[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:38:41 dist-test-slave-zkpd krb5kdc[18730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:38:41 dist-test-slave-zkpd krb5kdc[18730](info): set up 2 sockets
Aug 11 07:38:41 dist-test-slave-zkpd krb5kdc[18730](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 07:38:42.075217 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:38:42 dist-test-slave-zkpd krb5kdc[18730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:38:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:38:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:38:42.198544 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41593
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:41593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:42.312672 18746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:42.312970 18746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:42.313054 18746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:42.317098 18746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:38:42.317178 18746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:42.317211 18746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:42.317247 18746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:38:42.317282 18746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:38:42.322278 18746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:41593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:42.323320 18746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:42.324436 18746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:38:42.333120 18746 server_base.cc:1034] running on GCE node
W20260811 07:38:42.333072 18753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:42.333072 18752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:42.333077 18755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:42.333920 18746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:42.334923 18746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:42.336076 18746 hybrid_clock.cc:648] HybridClock initialized: now 1786433922336061 us; error 43 us; skew 500 ppm
Aug 11 07:38:42 dist-test-slave-zkpd krb5kdc[18730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433922, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:42.343600 18746 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:38:42.344519 18746 webserver.cc:460] Webserver started at http://127.18.72.254:40707/ using document root <none> and password file <none>
I20260811 07:38:42.345079 18746 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:42.345139 18746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:42.345379 18746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:42.347426 18746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "36f36423021d4f5b9ba8470e9a8d8586"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "9132f5bebc1840a04d5a417e17e4ac49"
server_key_iv: "1ede45bff7aab72c45c29ec1fbfdac19"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.348035 18746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "36f36423021d4f5b9ba8470e9a8d8586"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "9132f5bebc1840a04d5a417e17e4ac49"
server_key_iv: "1ede45bff7aab72c45c29ec1fbfdac19"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.351630 18746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:38:42.354178 18762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:42.355383 18746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 07:38:42.355520 18746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "36f36423021d4f5b9ba8470e9a8d8586"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "9132f5bebc1840a04d5a417e17e4ac49"
server_key_iv: "1ede45bff7aab72c45c29ec1fbfdac19"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.355628 18746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:42.372175 18746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:42.374982 18746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:42.375176 18746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:42.382836 18746 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:41593
I20260811 07:38:42.382850 18819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:41593 every 8 connection(s)
I20260811 07:38:42.383740 18746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:38:42.384555 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 18746
I20260811 07:38:42.384711 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:38:42.385987 18723 external_mini_cluster.cc:1428] Setting key bb18df9496326a8a67706b543dce8663
I20260811 07:38:42.386739 18820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:42.393601 18820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586: Bootstrap starting.
I20260811 07:38:42.396183 18820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:42.397125 18820 log.cc:824] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:42.399098 18820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586: No bootstrap required, opened a new log
I20260811 07:38:42.403630 18820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f36423021d4f5b9ba8470e9a8d8586" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41593 } }
I20260811 07:38:42.403934 18820 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:42.403997 18820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f36423021d4f5b9ba8470e9a8d8586, State: Initialized, Role: FOLLOWER
I20260811 07:38:42.404520 18820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f36423021d4f5b9ba8470e9a8d8586" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41593 } }
I20260811 07:38:42.404695 18820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:38:42.404772 18820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:38:42.404887 18820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 07:38:42 dist-test-slave-zkpd krb5kdc[18730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:42.405829 18820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f36423021d4f5b9ba8470e9a8d8586" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41593 } }
I20260811 07:38:42.406198 18820 leader_election.cc:302] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36f36423021d4f5b9ba8470e9a8d8586; no voters:
I20260811 07:38:42.406565 18820 leader_election.cc:288] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:38:42.406675 18826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:42.406926 18826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [term 1 LEADER]: Becoming Leader. State: Replica: 36f36423021d4f5b9ba8470e9a8d8586, State: Running, Role: LEADER
I20260811 07:38:42.407217 18826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f36423021d4f5b9ba8470e9a8d8586" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41593 } }
I20260811 07:38:42.408028 18820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:38:42.408438 18826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36f36423021d4f5b9ba8470e9a8d8586. Latest consensus state: current_term: 1 leader_uuid: "36f36423021d4f5b9ba8470e9a8d8586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f36423021d4f5b9ba8470e9a8d8586" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41593 } } }
I20260811 07:38:42.408561 18826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:42.409114 18827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36f36423021d4f5b9ba8470e9a8d8586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f36423021d4f5b9ba8470e9a8d8586" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41593 } } }
I20260811 07:38:42.409264 18827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586 [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:42.412793 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:42.389155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36736 (local address 127.18.72.254:41593)
0811 07:38:42.389626 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:42.389646 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:42.390682 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 07:38:42.394765 (+ 4083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:42.394780 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:42.395070 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:42.395339 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:42.398246 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.399352 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.400976 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.401443 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.401556 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:42.406238 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:42.406244 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:42.406259 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:42.406290 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:42.408500 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:42.409392 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:42.409397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:42.409402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:42.409530 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:42.409935 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:42.409941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:42.409944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:42.410499 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:42.410726 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:42.411574 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 07:38:42.411848 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":150,"threads_started":1}
W20260811 07:38:42.413426 18843 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:38:42.413511 18843 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:38:42.413578 18830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:38:42.414469 18830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:38:42.419929 18830 catalog_manager.cc:1347] Generated new cluster ID: 682cf7b7294949949d9ccf7fe75e2ab6
I20260811 07:38:42.419988 18830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:38:42.436362 18830 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:38:42.437299 18830 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:38:42.453068 18830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36f36423021d4f5b9ba8470e9a8d8586: Generated new TSK 0
I20260811 07:38:42.453799 18830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:38:42.488278 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41593
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:42.599792 18851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:42.600054 18851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:42.600136 18851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:42.604296 18851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:42.604348 18851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:42.604406 18851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:38:42.609856 18851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:42.610889 18851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:42.611933 18851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:42.621301 18857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:42.621315 18856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:42.621295 18859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:42.621789 18851 server_base.cc:1034] running on GCE node
I20260811 07:38:42.622139 18851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:42.622677 18851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:42.623837 18851 hybrid_clock.cc:648] HybridClock initialized: now 1786433922623817 us; error 29 us; skew 500 ppm
Aug 11 07:38:42 dist-test-slave-zkpd krb5kdc[18730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433922, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:42.626622 18851 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:38:42.627528 18851 webserver.cc:460] Webserver started at http://127.18.72.193:45169/ using document root <none> and password file <none>
I20260811 07:38:42.628111 18851 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:42.628173 18851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:42.628334 18851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:42.630201 18851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "d0a0f15211d448fdaff2a26379db52ea"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "e6c0c411a8a8cf6158ba57c405d8936c"
server_key_iv: "a5291c836263779b7b54189eb7b41886"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.630695 18851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "d0a0f15211d448fdaff2a26379db52ea"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "e6c0c411a8a8cf6158ba57c405d8936c"
server_key_iv: "a5291c836263779b7b54189eb7b41886"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.634265 18851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 07:38:42.636520 18867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:42.637665 18851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:38:42.637845 18851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "d0a0f15211d448fdaff2a26379db52ea"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "e6c0c411a8a8cf6158ba57c405d8936c"
server_key_iv: "a5291c836263779b7b54189eb7b41886"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.637955 18851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:42.668443 18851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:42.671408 18851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:42.671617 18851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:42.672312 18851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:42.673187 18851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:42.673231 18851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:42.673272 18851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:42.673328 18851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:42.682734 18851 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:41895
I20260811 07:38:42.682767 18992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:41895 every 8 connection(s)
I20260811 07:38:42.683627 18851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:38:42.684391 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 18851
I20260811 07:38:42.684478 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:38:42.684837 18723 external_mini_cluster.cc:1428] Setting key cceaee3b8282e54b72907dee2ff2b946
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Aug 11 07:38:42 dist-test-slave-zkpd krb5kdc[18730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433922, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
I20260811 07:38:42.701493 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:42.685656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:55867 (local address 127.18.72.254:41593)
0811 07:38:42.685793 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:42.685797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:42.688444 (+ 2647us) server_negotiation.cc:407] Connection header received
0811 07:38:42.690014 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:42.690018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:42.690072 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:42.690143 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:42.691944 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.692483 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.694389 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.694610 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.694682 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:42.697502 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:42.697505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:42.697507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:42.697531 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:42.698839 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:42.699462 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:42.699468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:42.699473 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:42.699581 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:42.700234 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:42.700238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:42.700240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:42.700452 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:42.700575 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:42.701137 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 07:38:42.701296 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:38:42.702312 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:42.686188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41593 (local address 127.18.72.193:55867)
0811 07:38:42.688248 (+ 2060us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:42.688318 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:42.689724 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:42.690374 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:42.690386 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:42.690946 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:42.691737 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:42.691765 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.692629 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.692633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:42.694252 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:42.694269 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.694781 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.694788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:42.695024 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:42.695627 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:42.695651 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:42.697342 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:42.698931 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:42.698936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:42.698947 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:42.699231 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:42.699678 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:42.699685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:42.699690 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:42.700083 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:42.700588 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:42.700597 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:42.700853 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 07:38:42.701070 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 07:38:42.701434 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1809,"thread_start_us":149,"threads_started":1}
I20260811 07:38:42.703689 18993 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41593
I20260811 07:38:42.704047 18993 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:42.704581 18993 heartbeater.cc:511] Master 127.18.72.254:41593 requested a full tablet report, sending...
I20260811 07:38:42.706185 18781 ts_manager.cc:194] Registered new tserver with Master: d0a0f15211d448fdaff2a26379db52ea (127.18.72.193:41895)
I20260811 07:38:42.707454 18781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:55867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:38:42.715055 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41593
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:42.829181 18998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:42.829425 18998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:42.829502 18998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:42.833560 18998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:42.833611 18998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:42.833737 18998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:38:42.838860 18998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:42.839882 18998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:42.840925 18998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:42.850389 19006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:42.850363 19004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:42.850514 18998 server_base.cc:1034] running on GCE node
W20260811 07:38:42.850705 19003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:42.851127 18998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:42.851759 18998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:42.852958 18998 hybrid_clock.cc:648] HybridClock initialized: now 1786433922852943 us; error 32 us; skew 500 ppm
Aug 11 07:38:42 dist-test-slave-zkpd krb5kdc[18730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433922, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:42.855578 18998 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:38:42.856442 18998 webserver.cc:460] Webserver started at http://127.18.72.194:45995/ using document root <none> and password file <none>
I20260811 07:38:42.856988 18998 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:42.857080 18998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:42.857311 18998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:42.859339 18998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "d8f39c5cdad648b6a2546b2bdc508510"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "79abc0f6c29676e24df73bbd65c7c706"
server_key_iv: "6773dcd66d3d2706ab0fb27b1acabbea"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.859951 18998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "d8f39c5cdad648b6a2546b2bdc508510"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "79abc0f6c29676e24df73bbd65c7c706"
server_key_iv: "6773dcd66d3d2706ab0fb27b1acabbea"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.863570 18998 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:38:42.865912 19013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:42.867065 18998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:38:42.867199 18998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "d8f39c5cdad648b6a2546b2bdc508510"
format_stamp: "Formatted at 2026-08-11 07:38:42 on dist-test-slave-zkpd"
server_key: "79abc0f6c29676e24df73bbd65c7c706"
server_key_iv: "6773dcd66d3d2706ab0fb27b1acabbea"
server_key_version: "encryptionkey@0"
I20260811 07:38:42.867305 18998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:42.912698 18998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:42.915633 18998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:42.915849 18998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:42.916477 18998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:42.917332 18998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:42.917404 18998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:42.917475 18998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:42.917528 18998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:42.926838 18998 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:41649
I20260811 07:38:42.926857 19137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:41649 every 8 connection(s)
I20260811 07:38:42.927743 18998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:38:42.932751 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 18998
I20260811 07:38:42.932842 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:38:42.933080 18723 external_mini_cluster.cc:1428] Setting key 5381eadce8bc5cc867dd11974feded2c
Aug 11 07:38:42 dist-test-slave-zkpd krb5kdc[18730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433922, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:42.941888 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:42.929648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:42165 (local address 127.18.72.254:41593)
0811 07:38:42.929857 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:42.929862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:42.930682 (+ 820us) server_negotiation.cc:407] Connection header received
0811 07:38:42.931981 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:42.931984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:42.932032 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:42.932107 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:42.933366 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.934016 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.935338 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.935551 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.935662 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:42.938210 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:42.938213 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:42.938215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:42.938236 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:42.939664 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:42.940237 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:42.940242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:42.940244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:42.940326 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:42.940676 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:42.940680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:42.940682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:42.940909 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:42.941013 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:42.941588 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 07:38:42.941706 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 07:38:42.942688 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:42.930037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41593 (local address 127.18.72.194:42165)
0811 07:38:42.930531 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:42.930573 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:42.931730 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:42.932274 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:42.932281 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:42.932635 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:42.933222 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:42.933238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.934145 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.934148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:42.935202 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:42.935212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:42.935701 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:42.935705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:42.935883 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:42.936576 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:42.936593 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:42.938041 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:42.939782 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:42.939789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:42.939804 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:42.940137 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:42.940420 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:42.940423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:42.940425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:42.940568 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:42.941041 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:42.941069 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:42.941341 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 07:38:42.941528 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 07:38:42.941871 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:38:42.943938 19138 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41593
I20260811 07:38:42.944190 19138 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:42.944741 19138 heartbeater.cc:511] Master 127.18.72.254:41593 requested a full tablet report, sending...
I20260811 07:38:42.945881 18781 ts_manager.cc:194] Registered new tserver with Master: d8f39c5cdad648b6a2546b2bdc508510 (127.18.72.194:41649)
I20260811 07:38:42.946484 18781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:42165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:38:42.960688 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41593
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:43.072702 19143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:43.072933 19143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:43.072974 19143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:43.076799 19143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:43.076853 19143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:43.076908 19143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:38:43.081836 19143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41593
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:43.082739 19143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:43.083671 19143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:43.092994 19148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:43.093000 19151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:43.093094 19149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:43.093152 19143 server_base.cc:1034] running on GCE node
I20260811 07:38:43.093663 19143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:43.094285 19143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:43.095460 19143 hybrid_clock.cc:648] HybridClock initialized: now 1786433923095429 us; error 40 us; skew 500 ppm
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[18730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433923, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:43.098109 19143 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:38:43.098958 19143 webserver.cc:460] Webserver started at http://127.18.72.195:37611/ using document root <none> and password file <none>
I20260811 07:38:43.099517 19143 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:43.099607 19143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:43.099838 19143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:43.101770 19143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "0ab8cc1292b14649b58c8ca445e25ec9"
format_stamp: "Formatted at 2026-08-11 07:38:43 on dist-test-slave-zkpd"
server_key: "f7efe24820673c165de234301d26e3d0"
server_key_iv: "d35a3dd6c7b255931a09c0829c8dbd3b"
server_key_version: "encryptionkey@0"
I20260811 07:38:43.102398 19143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "0ab8cc1292b14649b58c8ca445e25ec9"
format_stamp: "Formatted at 2026-08-11 07:38:43 on dist-test-slave-zkpd"
server_key: "f7efe24820673c165de234301d26e3d0"
server_key_iv: "d35a3dd6c7b255931a09c0829c8dbd3b"
server_key_version: "encryptionkey@0"
I20260811 07:38:43.106027 19143 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:38:43.108362 19158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:43.109592 19143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:38:43.109727 19143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "0ab8cc1292b14649b58c8ca445e25ec9"
format_stamp: "Formatted at 2026-08-11 07:38:43 on dist-test-slave-zkpd"
server_key: "f7efe24820673c165de234301d26e3d0"
server_key_iv: "d35a3dd6c7b255931a09c0829c8dbd3b"
server_key_version: "encryptionkey@0"
I20260811 07:38:43.109843 19143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:43.123667 19143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:43.126288 19143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:43.126443 19143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:43.126948 19143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:43.127768 19143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:43.127874 19143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:43.127974 19143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:43.128007 19143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:43.137334 19143 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:34837
I20260811 07:38:43.137359 19281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:34837 every 8 connection(s)
I20260811 07:38:43.138177 19143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:38:43.146440 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 19143
I20260811 07:38:43.146579 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:38:43.146898 18723 external_mini_cluster.cc:1428] Setting key ddc5c8620a4d163c77c81e1a370cc9fa
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[18730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433923, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:43.152104 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:43.139885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:46041 (local address 127.18.72.254:41593)
0811 07:38:43.140044 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:43.140047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:43.140868 (+ 821us) server_negotiation.cc:407] Connection header received
0811 07:38:43.142254 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:43.142258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:43.142330 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:43.142459 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:43.143595 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:43.144187 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:43.145305 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:43.145554 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:43.145662 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:43.148417 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:43.148421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:43.148425 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:43.148453 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:43.150063 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:43.150574 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:43.150578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:43.150580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:43.150665 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:43.151012 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:43.151017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:43.151019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:43.151199 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:43.151313 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:43.151788 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 07:38:43.151949 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 07:38:43.152853 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:43.140287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41593 (local address 127.18.72.195:46041)
0811 07:38:43.140717 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:43.140756 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:43.142037 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:43.142608 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:43.142615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:43.142962 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:43.143422 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:43.143436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:43.144310 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:43.144316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:43.145170 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:43.145179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:43.145712 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:43.145716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:43.145882 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:43.146437 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:43.146462 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:43.148250 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:43.150186 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:43.150192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:43.150205 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:43.150468 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:43.150766 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:43.150769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:43.150771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:43.150915 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:43.151312 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:43.151319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:43.151560 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 07:38:43.151744 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 07:38:43.152108 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":117,"threads_started":1}
I20260811 07:38:43.153868 19282 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41593
I20260811 07:38:43.154100 19282 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:43.154552 19282 heartbeater.cc:511] Master 127.18.72.254:41593 requested a full tablet report, sending...
I20260811 07:38:43.155494 18781 ts_manager.cc:194] Registered new tserver with Master: 0ab8cc1292b14649b58c8ca445e25ec9 (127.18.72.195:34837)
I20260811 07:38:43.156101 18781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:46041
I20260811 07:38:43.161121 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[18730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[18730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:38:43.187418 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:43.178686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.18.72.193:41895)
0811 07:38:43.179012 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:43.179037 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:43.179060 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:38:43.179110 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:43.179115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:43.179316 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:43.179440 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:43.180188 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:43.180957 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:43.181727 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:43.182079 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:43.182158 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:43.184434 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:43.184443 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:43.184456 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:43.184480 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:43.185903 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:43.186298 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:43.186303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:43.186304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:43.186383 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:43.186609 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:43.186612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:43.186613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:43.186836 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:43.187000 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:43.187138 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 07:38:43.187256 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":145,"threads_started":1}
W20260811 07:38:43.188445 18896 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49106
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[18730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[18730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:38:43.214120 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:43.205653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.18.72.193:41895)
0811 07:38:43.205803 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:43.205807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:43.205945 (+ 138us) server_negotiation.cc:407] Connection header received
0811 07:38:43.206116 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:43.206119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:43.206160 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:43.206244 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:43.206914 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:43.207467 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:43.208412 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:43.208699 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:43.208840 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:43.211292 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:43.211295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:43.211297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:43.211320 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:43.212584 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:43.213003 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:43.213005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:43.213007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:43.213078 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:43.213388 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:43.213390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:43.213391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:43.213551 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:43.213648 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:43.213906 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 07:38:43.214000 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:38:43.215107 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 18851
I20260811 07:38:43.224135 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 18998
I20260811 07:38:43.230858 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 19143
I20260811 07:38:43.237365 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 18746
2026-08-11T07:38:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2269 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[19296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[19296](info): set up 2 sockets
Aug 11 07:38:43 dist-test-slave-zkpd krb5kdc[19296](info): commencing operation
krb5kdc: starting...
W20260811 07:38:44.334189 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:38:44 dist-test-slave-zkpd krb5kdc[19296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:38:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:38:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:38:44.454809 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41117
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:41117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:44.566001 19310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:44.566287 19310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:44.566366 19310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:44.570297 19310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:38:44.570376 19310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:44.570416 19310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:44.570474 19310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:38:44.570513 19310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:38:44.575595 19310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:41117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:44.576728 19310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:44.577737 19310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:44.586256 19316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:44.586229 19318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:44.586248 19315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:44.586431 19310 server_base.cc:1034] running on GCE node
I20260811 07:38:44.587070 19310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:44.588189 19310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:44.589388 19310 hybrid_clock.cc:648] HybridClock initialized: now 1786433924589375 us; error 23 us; skew 500 ppm
Aug 11 07:38:44 dist-test-slave-zkpd krb5kdc[19296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433924, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:44.592655 19310 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:38:44.593495 19310 webserver.cc:460] Webserver started at http://127.18.72.254:33011/ using document root <none> and password file <none>
I20260811 07:38:44.594065 19310 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:44.594157 19310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:44.594414 19310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:44.596503 19310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "ca6f86636aee4d80be5d140d2e63f10a"
format_stamp: "Formatted at 2026-08-11 07:38:44 on dist-test-slave-zkpd"
server_key: "1732e11e987c80814ebe8365dedc719a"
server_key_iv: "b7ba9a9611aeda7e43f8f2f66bfe24e3"
server_key_version: "encryptionkey@0"
I20260811 07:38:44.597074 19310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "ca6f86636aee4d80be5d140d2e63f10a"
format_stamp: "Formatted at 2026-08-11 07:38:44 on dist-test-slave-zkpd"
server_key: "1732e11e987c80814ebe8365dedc719a"
server_key_iv: "b7ba9a9611aeda7e43f8f2f66bfe24e3"
server_key_version: "encryptionkey@0"
I20260811 07:38:44.600814 19310 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:38:44.603163 19327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:44.604422 19310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:38:44.604570 19310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "ca6f86636aee4d80be5d140d2e63f10a"
format_stamp: "Formatted at 2026-08-11 07:38:44 on dist-test-slave-zkpd"
server_key: "1732e11e987c80814ebe8365dedc719a"
server_key_iv: "b7ba9a9611aeda7e43f8f2f66bfe24e3"
server_key_version: "encryptionkey@0"
I20260811 07:38:44.604681 19310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:44.618654 19310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:44.621886 19310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:44.622093 19310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:44.629209 19310 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:41117
I20260811 07:38:44.629225 19383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:41117 every 8 connection(s)
I20260811 07:38:44.630209 19310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:38:44.630543 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 19310
I20260811 07:38:44.630659 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:38:44.630997 18723 external_mini_cluster.cc:1428] Setting key 3d18cb34b256aaab6494a94ff4f65bb0
I20260811 07:38:44.633610 19384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:44.639168 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a: Bootstrap starting.
Aug 11 07:38:44 dist-test-slave-zkpd krb5kdc[19296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:44.641677 19384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:44.642416 19384 log.cc:824] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:44.644357 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a: No bootstrap required, opened a new log
I20260811 07:38:44.644752 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:44.632413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.18.72.254:41117)
0811 07:38:44.632855 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:44.632870 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:44.632904 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:38:44.633538 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:44.633553 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:44.633945 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:44.634273 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:44.635078 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:44.636146 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:44.637005 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:44.637469 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:44.637651 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:44.640196 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:44.640201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:44.640214 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:44.640240 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:44.641926 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:44.642357 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:44.642364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:44.642370 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:44.642492 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:44.642798 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:44.642802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:44.642803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:44.643182 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:44.643384 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:44.643690 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 07:38:44.644024 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":141,"threads_started":1}
I20260811 07:38:44.648003 19384 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f86636aee4d80be5d140d2e63f10a" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41117 } }
I20260811 07:38:44.648238 19384 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:44.648335 19384 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6f86636aee4d80be5d140d2e63f10a, State: Initialized, Role: FOLLOWER
I20260811 07:38:44.648793 19384 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f86636aee4d80be5d140d2e63f10a" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41117 } }
I20260811 07:38:44.648942 19384 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:38:44.649029 19384 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:38:44.649129 19384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:44.649991 19384 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f86636aee4d80be5d140d2e63f10a" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41117 } }
I20260811 07:38:44.650336 19384 leader_election.cc:302] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca6f86636aee4d80be5d140d2e63f10a; no voters:
I20260811 07:38:44.650676 19384 leader_election.cc:288] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:38:44.650745 19389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:44.650959 19389 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [term 1 LEADER]: Becoming Leader. State: Replica: ca6f86636aee4d80be5d140d2e63f10a, State: Running, Role: LEADER
I20260811 07:38:44.651288 19389 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f86636aee4d80be5d140d2e63f10a" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41117 } }
I20260811 07:38:44.651930 19384 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:38:44.653226 19391 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca6f86636aee4d80be5d140d2e63f10a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f86636aee4d80be5d140d2e63f10a" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41117 } } }
I20260811 07:38:44.653358 19391 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:44.654146 19398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:38:44.654119 19390 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca6f86636aee4d80be5d140d2e63f10a. Latest consensus state: current_term: 1 leader_uuid: "ca6f86636aee4d80be5d140d2e63f10a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f86636aee4d80be5d140d2e63f10a" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41117 } } }
I20260811 07:38:44.654239 19390 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:44.656879 19398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:38:44.661993 19398 catalog_manager.cc:1347] Generated new cluster ID: 1190dd8479df4f619883b9a0e233a0c6
I20260811 07:38:44.662053 19398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:38:44.676473 19398 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:38:44.677695 19398 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:38:44.682591 19398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca6f86636aee4d80be5d140d2e63f10a: Generated new TSK 0
I20260811 07:38:44.683298 19398 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:38:44.712652 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41117
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:44.826223 19410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:44.826493 19410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:44.826566 19410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:44.830613 19410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:44.830693 19410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:44.830782 19410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:38:44.835911 19410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:44.836894 19410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:44.837888 19410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:44.847255 19416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:44.847242 19417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:44.847271 19419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:44.847445 19410 server_base.cc:1034] running on GCE node
I20260811 07:38:44.847880 19410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:44.848582 19410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:44.849740 19410 hybrid_clock.cc:648] HybridClock initialized: now 1786433924849720 us; error 30 us; skew 500 ppm
Aug 11 07:38:44 dist-test-slave-zkpd krb5kdc[19296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433924, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:44.852545 19410 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:38:44.853447 19410 webserver.cc:460] Webserver started at http://127.18.72.193:43329/ using document root <none> and password file <none>
I20260811 07:38:44.854023 19410 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:44.854084 19410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:44.854241 19410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:44.856230 19410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "aa4b0312f8a049de98a9229d5b3c4888"
format_stamp: "Formatted at 2026-08-11 07:38:44 on dist-test-slave-zkpd"
server_key: "c9185068c5f07e5216366a046248dfbc"
server_key_iv: "bbdf5f8134c7253b78a2278fd3d59343"
server_key_version: "encryptionkey@0"
I20260811 07:38:44.856736 19410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "aa4b0312f8a049de98a9229d5b3c4888"
format_stamp: "Formatted at 2026-08-11 07:38:44 on dist-test-slave-zkpd"
server_key: "c9185068c5f07e5216366a046248dfbc"
server_key_iv: "bbdf5f8134c7253b78a2278fd3d59343"
server_key_version: "encryptionkey@0"
I20260811 07:38:44.860358 19410 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 07:38:44.862679 19427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:44.863826 19410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:38:44.863993 19410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "aa4b0312f8a049de98a9229d5b3c4888"
format_stamp: "Formatted at 2026-08-11 07:38:44 on dist-test-slave-zkpd"
server_key: "c9185068c5f07e5216366a046248dfbc"
server_key_iv: "bbdf5f8134c7253b78a2278fd3d59343"
server_key_version: "encryptionkey@0"
I20260811 07:38:44.864094 19410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:44.894902 19410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:44.898350 19410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:44.898559 19410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:44.899228 19410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:44.900164 19410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:44.900213 19410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:44.900281 19410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:44.900313 19410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:44.909339 19410 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:44017
I20260811 07:38:44.909366 19552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:44017 every 8 connection(s)
I20260811 07:38:44.910202 19410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:38:44 dist-test-slave-zkpd krb5kdc[19296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433924, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:44.919492 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 19410
I20260811 07:38:44.919611 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:38:44.919881 18723 external_mini_cluster.cc:1428] Setting key e3327a42efda54783c1c402e4862f596
I20260811 07:38:44.923548 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:44.911962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:39053 (local address 127.18.72.254:41117)
0811 07:38:44.912116 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:44.912121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:44.912959 (+ 838us) server_negotiation.cc:407] Connection header received
0811 07:38:44.914321 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:44.914324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:44.914373 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:44.914446 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:44.915547 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:44.916167 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:44.917232 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:44.917446 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:44.917518 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:44.919880 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:44.919884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:44.919913 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:44.919943 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:44.921555 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:44.922058 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:44.922062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:44.922064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:44.922143 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:44.922492 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:44.922497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:44.922498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:44.922676 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:44.922814 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:44.923337 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 07:38:44.923437 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:38:44.924710 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:44.912360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41117 (local address 127.18.72.193:39053)
0811 07:38:44.912816 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:44.912857 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:44.914125 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:44.914602 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:44.914609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:44.914942 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:44.915413 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:44.915426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:44.916273 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:44.916288 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:44.917128 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:44.917137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:44.917579 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:44.917582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:44.917717 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:44.918222 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:44.918237 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:44.919717 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:44.921671 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:44.921677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:44.921690 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:44.921950 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:44.922239 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:44.922243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:44.922245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:44.922390 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:44.922785 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:44.922791 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:44.923029 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 07:38:44.923264 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 07:38:44.923621 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":123,"threads_started":1}
I20260811 07:38:44.925926 19553 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41117
I20260811 07:38:44.926189 19553 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:44.926810 19553 heartbeater.cc:511] Master 127.18.72.254:41117 requested a full tablet report, sending...
I20260811 07:38:44.928418 19345 ts_manager.cc:194] Registered new tserver with Master: aa4b0312f8a049de98a9229d5b3c4888 (127.18.72.193:44017)
I20260811 07:38:44.929527 19345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:39053
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:38:44.948819 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41117
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:45.060730 19559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:45.060983 19559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:45.061025 19559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:45.064891 19559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:45.064937 19559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:45.064991 19559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:38:45.070428 19559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:45.071319 19559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:45.072312 19559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:45.081955 19564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:45.081837 19565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:45.082013 19559 server_base.cc:1034] running on GCE node
W20260811 07:38:45.081836 19567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:45.082538 19559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:45.083155 19559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:45.084307 19559 hybrid_clock.cc:648] HybridClock initialized: now 1786433925084287 us; error 30 us; skew 500 ppm
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433925, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:45.087055 19559 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:38:45.088016 19559 webserver.cc:460] Webserver started at http://127.18.72.194:36343/ using document root <none> and password file <none>
I20260811 07:38:45.088584 19559 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:45.088644 19559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:45.088867 19559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:45.090931 19559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "8f48ab0099d24c17bc127a7d723e93f4"
format_stamp: "Formatted at 2026-08-11 07:38:45 on dist-test-slave-zkpd"
server_key: "f6da344869d1218b544648e84e50d4c6"
server_key_iv: "eed83de7cd6d58d914f7049a26cd1490"
server_key_version: "encryptionkey@0"
I20260811 07:38:45.091529 19559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "8f48ab0099d24c17bc127a7d723e93f4"
format_stamp: "Formatted at 2026-08-11 07:38:45 on dist-test-slave-zkpd"
server_key: "f6da344869d1218b544648e84e50d4c6"
server_key_iv: "eed83de7cd6d58d914f7049a26cd1490"
server_key_version: "encryptionkey@0"
I20260811 07:38:45.095150 19559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 07:38:45.097571 19577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:45.098758 19559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:38:45.098994 19559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "8f48ab0099d24c17bc127a7d723e93f4"
format_stamp: "Formatted at 2026-08-11 07:38:45 on dist-test-slave-zkpd"
server_key: "f6da344869d1218b544648e84e50d4c6"
server_key_iv: "eed83de7cd6d58d914f7049a26cd1490"
server_key_version: "encryptionkey@0"
I20260811 07:38:45.099120 19559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:45.116812 19559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:45.119638 19559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:45.119853 19559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:45.120474 19559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:45.121320 19559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:45.121390 19559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:45.121460 19559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:45.121503 19559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:45.130584 19559 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:38163
I20260811 07:38:45.130615 19703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:38163 every 8 connection(s)
I20260811 07:38:45.131469 19559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:38:45.135113 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 19559
I20260811 07:38:45.135210 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:38:45.135486 18723 external_mini_cluster.cc:1428] Setting key dcf01e6243fb0ba17e6c62c2647afeec
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433925, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:45.145895 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.133281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:53049 (local address 127.18.72.254:41117)
0811 07:38:45.133418 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.133422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.134253 (+ 831us) server_negotiation.cc:407] Connection header received
0811 07:38:45.135737 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.135741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.135795 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.135881 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:45.137267 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.137780 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.139058 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.139280 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.139362 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.142205 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:45.142209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:45.142211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:45.142233 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:45.143799 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.144356 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.144360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.144362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.144451 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.144815 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.144819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.144820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.144999 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:45.145126 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.145601 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.145730 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:38:45.146714 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.133638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41117 (local address 127.18.72.194:53049)
0811 07:38:45.134098 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:45.134140 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:45.135478 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:45.136102 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:45.136110 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:45.136583 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:45.137121 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.137135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.137915 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.137923 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.138902 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.138917 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.139436 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.139442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.139662 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.140349 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:45.140375 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:45.142031 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:45.143960 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:45.143966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:45.143980 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:45.144246 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:45.144547 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:45.144550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:45.144552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:45.144701 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:45.145124 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:45.145130 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:45.145365 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 07:38:45.145541 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 07:38:45.145848 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":121,"threads_started":1}
I20260811 07:38:45.147941 19704 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41117
I20260811 07:38:45.148180 19704 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:45.148691 19704 heartbeater.cc:511] Master 127.18.72.254:41117 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
I20260811 07:38:45.149864 19345 ts_manager.cc:194] Registered new tserver with Master: 8f48ab0099d24c17bc127a7d723e93f4 (127.18.72.194:38163)
I20260811 07:38:45.150449 19345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:53049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:38:45.166291 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41117
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:45.279048 19709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:45.279289 19709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:45.279331 19709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:45.283370 19709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:45.283419 19709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:45.283509 19709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:38:45.288697 19709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41117
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:45.289753 19709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:45.290959 19709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:45.300915 19721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:45.300906 19717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:45.300908 19718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:45.301066 19709 server_base.cc:1034] running on GCE node
I20260811 07:38:45.301694 19709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:45.302435 19709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:45.303632 19709 hybrid_clock.cc:648] HybridClock initialized: now 1786433925303617 us; error 27 us; skew 500 ppm
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433925, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:45.306625 19709 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:38:45.307447 19709 webserver.cc:460] Webserver started at http://127.18.72.195:39497/ using document root <none> and password file <none>
I20260811 07:38:45.308079 19709 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:45.308195 19709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:45.308429 19709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:45.310514 19709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "83d51d7fd37748e49028e0baf16fdb59"
format_stamp: "Formatted at 2026-08-11 07:38:45 on dist-test-slave-zkpd"
server_key: "773d94c1fd85436dc21919ce7b9ecdcf"
server_key_iv: "7595577459a77f81b899ffe3efa2799b"
server_key_version: "encryptionkey@0"
I20260811 07:38:45.311074 19709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "83d51d7fd37748e49028e0baf16fdb59"
format_stamp: "Formatted at 2026-08-11 07:38:45 on dist-test-slave-zkpd"
server_key: "773d94c1fd85436dc21919ce7b9ecdcf"
server_key_iv: "7595577459a77f81b899ffe3efa2799b"
server_key_version: "encryptionkey@0"
I20260811 07:38:45.314841 19709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 07:38:45.317548 19729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:45.318760 19709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:38:45.318856 19709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "83d51d7fd37748e49028e0baf16fdb59"
format_stamp: "Formatted at 2026-08-11 07:38:45 on dist-test-slave-zkpd"
server_key: "773d94c1fd85436dc21919ce7b9ecdcf"
server_key_iv: "7595577459a77f81b899ffe3efa2799b"
server_key_version: "encryptionkey@0"
I20260811 07:38:45.318933 19709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:45.335207 19709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:45.337845 19709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:45.337988 19709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:45.338503 19709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:45.339221 19709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:45.339265 19709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:45.339300 19709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:45.339314 19709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:45.347959 19709 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:44495
I20260811 07:38:45.347983 19851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:44495 every 8 connection(s)
I20260811 07:38:45.348791 19709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:38:45.352896 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 19709
I20260811 07:38:45.353034 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:38:45.353353 18723 external_mini_cluster.cc:1428] Setting key 5d17beebd7af6947e83333e451b4e7e5
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433925, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:45.362254 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.350586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:59993 (local address 127.18.72.254:41117)
0811 07:38:45.350721 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.350725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.351539 (+ 814us) server_negotiation.cc:407] Connection header received
0811 07:38:45.353070 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.353073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.353132 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.353237 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:45.354696 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.355302 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.356547 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.356760 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.356846 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.359067 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:45.359070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:45.359072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:45.359094 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:45.360523 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.360981 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.360983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.360985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.361055 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.361335 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.361337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.361338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.361496 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:45.361569 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.362033 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.362143 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:38:45.363027 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.350938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41117 (local address 127.18.72.195:59993)
0811 07:38:45.351356 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:45.351434 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:45.352855 (+ 1421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:45.353418 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:45.353424 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:45.353956 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:45.354509 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.354523 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.355433 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.355436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.356438 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.356449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.356900 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.356902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.357034 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.357522 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:45.357537 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:45.358922 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:45.360636 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:45.360641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:45.360651 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:45.360896 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:45.361133 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:45.361135 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:45.361136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:45.361256 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:45.361581 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:45.361586 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:45.361827 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 07:38:45.361986 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 07:38:45.362256 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260811 07:38:45.364121 19852 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41117
I20260811 07:38:45.364352 19852 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:45.364832 19852 heartbeater.cc:511] Master 127.18.72.254:41117 requested a full tablet report, sending...
I20260811 07:38:45.365833 19345 ts_manager.cc:194] Registered new tserver with Master: 83d51d7fd37748e49028e0baf16fdb59 (127.18.72.195:44495)
I20260811 07:38:45.366576 19345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:59993
I20260811 07:38:45.367249 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:38:45.377890 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.370627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55058 (local address 127.18.72.254:41117)
0811 07:38:45.370763 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.370769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.370942 (+ 173us) server_negotiation.cc:407] Connection header received
0811 07:38:45.371076 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.371079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.371118 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.371206 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:45.371909 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.372434 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.373258 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.373487 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.373593 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.374940 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:45.374944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:45.374947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:45.374970 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:45.376252 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.376666 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.376670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.376672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.376755 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.377031 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.377035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.377037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.377241 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:45.377384 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.377613 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.377723 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:38:45.382499 19344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:38:45.385540 19344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:45.399703 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.393846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34490 (local address 127.18.72.195:44495)
0811 07:38:45.394238 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.394251 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.394813 (+ 562us) server_negotiation.cc:407] Connection header received
0811 07:38:45.394918 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.394923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.395142 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.395301 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:45.395847 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.396726 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.398361 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.399026 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.399149 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.399306 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.399463 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.399550 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":172,"threads_started":1}
I20260811 07:38:45.399680 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.393653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:44495 (local address 127.0.0.1:34490)
0811 07:38:45.394446 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:45.394494 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:45.394665 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:45.395503 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:45.395509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:45.395533 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:45.395706 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.395715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.396853 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.396859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.398215 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.398226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.399149 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.399153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.399272 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.399313 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 07:38:45.399423 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 07:38:45.399541 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260811 07:38:45.399974 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.393850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:38163 (local address 127.0.0.1:55918)
0811 07:38:45.394684 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:45.394698 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:45.394802 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:45.395544 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:45.395547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:45.395564 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:45.395722 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.395728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.397243 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.397247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.398431 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.398439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.399489 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.399493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.399599 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.399624 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 07:38:45.399673 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 07:38:45.399750 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":109,"thread_start_us":111,"threads_started":1}
I20260811 07:38:45.400550 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.394369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:44017 (local address 127.0.0.1:48010)
0811 07:38:45.395247 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:45.395259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:45.395360 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:45.395975 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:45.395977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:45.395996 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:45.396070 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.396080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.397437 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.397443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.398588 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.398598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.400210 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.400214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.400291 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.400312 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 07:38:45.400373 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 07:38:45.400440 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":112,"threads_started":1}
I20260811 07:38:45.400640 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.394511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.18.72.194:38163)
0811 07:38:45.395075 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.395088 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.395110 (+ 22us) server_negotiation.cc:407] Connection header received
0811 07:38:45.395193 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.395199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.395393 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.395677 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:45.395854 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.397081 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.398543 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.399341 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.400225 (+ 884us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.400375 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.400423 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.400502 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":115,"threads_started":1}
I20260811 07:38:45.401017 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.394534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.18.72.193:44017)
0811 07:38:45.394960 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.394975 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.395462 (+ 487us) server_negotiation.cc:407] Connection header received
0811 07:38:45.395546 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.395551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.395719 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.395865 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:45.396203 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.397310 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.398721 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.399513 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.399606 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.400683 (+ 1077us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.400747 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.400887 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":131,"threads_started":1}
I20260811 07:38:45.403049 19781 tablet_service.cc:1467] Processing CreateTablet for tablet 01c2d08bad314dcebf0c4f3bab2208d2 (DEFAULT_TABLE table=test-table [id=9a1169a4b10944078b901be96fc7af08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:45.403380 19633 tablet_service.cc:1467] Processing CreateTablet for tablet 01c2d08bad314dcebf0c4f3bab2208d2 (DEFAULT_TABLE table=test-table [id=9a1169a4b10944078b901be96fc7af08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:45.403451 19481 tablet_service.cc:1467] Processing CreateTablet for tablet 01c2d08bad314dcebf0c4f3bab2208d2 (DEFAULT_TABLE table=test-table [id=9a1169a4b10944078b901be96fc7af08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:45.404270 19781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01c2d08bad314dcebf0c4f3bab2208d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:45.404372 19633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01c2d08bad314dcebf0c4f3bab2208d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:45.404502 19481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01c2d08bad314dcebf0c4f3bab2208d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:45.410933 19872 tablet_bootstrap.cc:492] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888: Bootstrap starting.
I20260811 07:38:45.412256 19874 tablet_bootstrap.cc:492] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4: Bootstrap starting.
I20260811 07:38:45.412254 19875 tablet_bootstrap.cc:492] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59: Bootstrap starting.
I20260811 07:38:45.413215 19872 tablet_bootstrap.cc:654] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:45.414325 19872 log.cc:824] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:45.414472 19875 tablet_bootstrap.cc:654] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:45.414984 19874 tablet_bootstrap.cc:654] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:45.415748 19875 log.cc:824] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:45.415748 19874 log.cc:824] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:45.416677 19872 tablet_bootstrap.cc:492] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888: No bootstrap required, opened a new log
I20260811 07:38:45.416889 19872 ts_tablet_manager.cc:1397] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 07:38:45.417693 19875 tablet_bootstrap.cc:492] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59: No bootstrap required, opened a new log
I20260811 07:38:45.417882 19875 ts_tablet_manager.cc:1397] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 07:38:45.418007 19874 tablet_bootstrap.cc:492] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4: No bootstrap required, opened a new log
I20260811 07:38:45.418195 19874 ts_tablet_manager.cc:1397] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 07:38:45.419965 19872 raft_consensus.cc:348] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.420208 19872 raft_consensus.cc:374] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:45.420261 19872 raft_consensus.cc:729] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4b0312f8a049de98a9229d5b3c4888, State: Initialized, Role: FOLLOWER
I20260811 07:38:45.420713 19872 consensus_queue.cc:260] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.421366 19553 heartbeater.cc:503] Master 127.18.72.254:41117 was elected leader, sending a full tablet report...
I20260811 07:38:45.421669 19872 ts_tablet_manager.cc:1428] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 07:38:45.421681 19875 raft_consensus.cc:348] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.421959 19875 raft_consensus.cc:374] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:45.422015 19875 raft_consensus.cc:729] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83d51d7fd37748e49028e0baf16fdb59, State: Initialized, Role: FOLLOWER
I20260811 07:38:45.422411 19875 consensus_queue.cc:260] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.423126 19874 raft_consensus.cc:348] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.423383 19875 ts_tablet_manager.cc:1428] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 07:38:45.423389 19874 raft_consensus.cc:374] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:45.423439 19874 raft_consensus.cc:729] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f48ab0099d24c17bc127a7d723e93f4, State: Initialized, Role: FOLLOWER
I20260811 07:38:45.423727 19852 heartbeater.cc:503] Master 127.18.72.254:41117 was elected leader, sending a full tablet report...
I20260811 07:38:45.423949 19874 consensus_queue.cc:260] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.424563 19704 heartbeater.cc:503] Master 127.18.72.254:41117 was elected leader, sending a full tablet report...
I20260811 07:38:45.425391 19874 ts_tablet_manager.cc:1428] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 07:38:45.431850 19880 raft_consensus.cc:482] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:45.432057 19880 raft_consensus.cc:504] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.432982 19880 leader_election.cc:288] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f48ab0099d24c17bc127a7d723e93f4 (127.18.72.194:38163), aa4b0312f8a049de98a9229d5b3c4888 (127.18.72.193:44017)
I20260811 07:38:45.437583 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.433270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:49347 (local address 127.18.72.194:38163)
0811 07:38:45.433388 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.433394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.433412 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:38:45.433570 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.433573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.433623 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.433715 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:45.434023 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.434802 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.436295 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.436899 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.437019 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.437069 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.437399 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.437466 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 07:38:45.437583 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.433344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:35199 (local address 127.18.72.193:44017)
0811 07:38:45.433467 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.433473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.433812 (+ 339us) server_negotiation.cc:407] Connection header received
0811 07:38:45.433908 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.433910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.433944 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.434037 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:45.434263 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.434963 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.436310 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.436902 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.437019 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.437069 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.437399 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.437466 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:38:45.437613 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.433180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:38163 (local address 127.18.72.195:49347)
0811 07:38:45.433319 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:45.433336 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:45.433448 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:45.433743 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:45.433747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:45.433772 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:45.433913 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.433919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.434903 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.434905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.436159 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.436170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.437019 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.437023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.437250 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.437289 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 07:38:45.437370 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 07:38:45.437467 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":89}
I20260811 07:38:45.437628 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.433180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:44017 (local address 127.18.72.195:35199)
0811 07:38:45.433738 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:45.433751 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:45.433836 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:45.434060 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:45.434062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:45.434076 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:45.434165 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.434168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.435052 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.435055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.436206 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:45.436216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.437019 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.437023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:45.437206 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.437290 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 07:38:45.437370 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 07:38:45.437466 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":285,"thread_start_us":130,"threads_started":1}
I20260811 07:38:45.438289 19504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01c2d08bad314dcebf0c4f3bab2208d2" candidate_uuid: "83d51d7fd37748e49028e0baf16fdb59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4b0312f8a049de98a9229d5b3c4888" is_pre_election: true
I20260811 07:38:45.438359 19655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01c2d08bad314dcebf0c4f3bab2208d2" candidate_uuid: "83d51d7fd37748e49028e0baf16fdb59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f48ab0099d24c17bc127a7d723e93f4" is_pre_election: true
I20260811 07:38:45.438575 19655 raft_consensus.cc:2454] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83d51d7fd37748e49028e0baf16fdb59 in term 0.
I20260811 07:38:45.438575 19504 raft_consensus.cc:2454] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83d51d7fd37748e49028e0baf16fdb59 in term 0.
I20260811 07:38:45.439044 19731 leader_election.cc:302] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83d51d7fd37748e49028e0baf16fdb59, 8f48ab0099d24c17bc127a7d723e93f4; no voters:
I20260811 07:38:45.439255 19880 raft_consensus.cc:2781] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:45.439354 19880 raft_consensus.cc:482] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:45.439412 19880 raft_consensus.cc:3037] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:45.440497 19880 raft_consensus.cc:504] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.440882 19880 leader_election.cc:288] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [CANDIDATE]: Term 1 election: Requested vote from peers 8f48ab0099d24c17bc127a7d723e93f4 (127.18.72.194:38163), aa4b0312f8a049de98a9229d5b3c4888 (127.18.72.193:44017)
I20260811 07:38:45.441203 19655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01c2d08bad314dcebf0c4f3bab2208d2" candidate_uuid: "83d51d7fd37748e49028e0baf16fdb59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f48ab0099d24c17bc127a7d723e93f4"
I20260811 07:38:45.441226 19504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01c2d08bad314dcebf0c4f3bab2208d2" candidate_uuid: "83d51d7fd37748e49028e0baf16fdb59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4b0312f8a049de98a9229d5b3c4888"
I20260811 07:38:45.441346 19655 raft_consensus.cc:3037] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:45.441361 19504 raft_consensus.cc:3037] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:45.442349 19655 raft_consensus.cc:2454] T 01c2d08bad314dcebf0c4f3bab2208d2 P 8f48ab0099d24c17bc127a7d723e93f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83d51d7fd37748e49028e0baf16fdb59 in term 1.
I20260811 07:38:45.442355 19504 raft_consensus.cc:2454] T 01c2d08bad314dcebf0c4f3bab2208d2 P aa4b0312f8a049de98a9229d5b3c4888 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83d51d7fd37748e49028e0baf16fdb59 in term 1.
I20260811 07:38:45.442667 19733 leader_election.cc:302] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83d51d7fd37748e49028e0baf16fdb59, aa4b0312f8a049de98a9229d5b3c4888; no voters:
I20260811 07:38:45.442844 19880 raft_consensus.cc:2781] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:45.443008 19880 raft_consensus.cc:686] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [term 1 LEADER]: Becoming Leader. State: Replica: 83d51d7fd37748e49028e0baf16fdb59, State: Running, Role: LEADER
I20260811 07:38:45.443322 19880 consensus_queue.cc:237] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } }
I20260811 07:38:45.446349 19345 catalog_manager.cc:5495] T 01c2d08bad314dcebf0c4f3bab2208d2 P 83d51d7fd37748e49028e0baf16fdb59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83d51d7fd37748e49028e0baf16fdb59 (127.18.72.195). New cstate: current_term: 1 leader_uuid: "83d51d7fd37748e49028e0baf16fdb59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83d51d7fd37748e49028e0baf16fdb59" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 44495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f48ab0099d24c17bc127a7d723e93f4" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa4b0312f8a049de98a9229d5b3c4888" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44017 } health_report { overall_health: UNKNOWN } } }
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:38:45.465006 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.455047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48018 (local address 127.18.72.193:44017)
0811 07:38:45.455183 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.455187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.455318 (+ 131us) server_negotiation.cc:407] Connection header received
0811 07:38:45.455402 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.455404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.455441 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.455504 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:45.456238 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.456700 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.457506 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.457696 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.457766 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.460942 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:45.460954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:45.460972 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:45.460997 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:45.463007 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.463488 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.463495 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.463497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.463591 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.463887 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.463916 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.463918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.464195 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:45.464354 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.464731 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.464857 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:38:45.496100 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.487503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48028 (local address 127.18.72.193:44017)
0811 07:38:45.487676 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.487681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.487834 (+ 153us) server_negotiation.cc:407] Connection header received
0811 07:38:45.488023 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.488026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.488073 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.488144 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:45.488855 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.489358 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.490116 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.490332 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.490406 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.492725 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:45.492729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:45.492731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:45.492753 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:45.494351 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.494775 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.494779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.494781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.494873 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.495161 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.495165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.495166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.495343 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:45.495474 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.495770 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.495930 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:38:45.521735 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:45.514037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.18.72.193:44017)
0811 07:38:45.514206 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:45.514211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:45.514286 (+ 75us) server_negotiation.cc:407] Connection header received
0811 07:38:45.514387 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:45.514389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:45.514429 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:45.514497 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:45.515233 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.515721 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.516558 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:45.516761 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:45.516841 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:45.519027 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:45.519030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:45.519032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:45.519053 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:45.520305 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.520692 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.520694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.520696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.520763 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:45.520994 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:45.520996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:45.520997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:45.521137 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:45.521233 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:45.521438 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 07:38:45.521524 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 07:38:45.523303 19457 tablet_service.cc:2908] Rejecting scan request for tablet 01c2d08bad314dcebf0c4f3bab2208d2: Uninitialized: clean time has not yet been initialized
I20260811 07:38:45.525310 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 19410
I20260811 07:38:45.533447 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 19559
I20260811 07:38:45.542837 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 19709
I20260811 07:38:45.549415 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 19310
2026-08-11T07:38:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2312 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19899](info): set up 2 sockets
Aug 11 07:38:45 dist-test-slave-zkpd krb5kdc[19899](info): commencing operation
krb5kdc: starting...
W20260811 07:38:46.616777 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:38:46 dist-test-slave-zkpd krb5kdc[19899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:38:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:38:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:38:46.740859 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:44301
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:45999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:44301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:46.853641 19913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:46.853896 19913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:46.853935 19913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:46.857841 19913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:38:46.857885 19913 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 07:38:46.857903 19913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:46.857920 19913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:46.857936 19913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:38:46.857954 19913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:38:46.862921 19913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:45999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:44301
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:44301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:46.863812 19913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:46.864792 19913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:46.872936 19918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:46.872965 19919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:46.873102 19913 server_base.cc:1034] running on GCE node
W20260811 07:38:46.872961 19921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:46.873679 19913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:46.874827 19913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:46.875964 19913 hybrid_clock.cc:648] HybridClock initialized: now 1786433926875949 us; error 53 us; skew 500 ppm
Aug 11 07:38:46 dist-test-slave-zkpd krb5kdc[19899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433926, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:46.878746 19913 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:38:46.879623 19913 webserver.cc:460] Webserver started at http://127.18.72.254:44065/ using document root <none> and password file <none>
I20260811 07:38:46.880244 19913 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:46.880352 19913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:46.880585 19913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:46.882570 19913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "98c317794dd84712a50b2d18b38711e0"
format_stamp: "Formatted at 2026-08-11 07:38:46 on dist-test-slave-zkpd"
server_key: "da1d43dfa3b18073d6fc2a1e6fbf5f0d"
server_key_iv: "33aaaf81b96d545dd08082035ddb486c"
server_key_version: "encryptionkey@0"
I20260811 07:38:46.883138 19913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "98c317794dd84712a50b2d18b38711e0"
format_stamp: "Formatted at 2026-08-11 07:38:46 on dist-test-slave-zkpd"
server_key: "da1d43dfa3b18073d6fc2a1e6fbf5f0d"
server_key_iv: "33aaaf81b96d545dd08082035ddb486c"
server_key_version: "encryptionkey@0"
I20260811 07:38:46.886850 19913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 07:38:46.889278 19929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:46.890470 19913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:38:46.890604 19913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "98c317794dd84712a50b2d18b38711e0"
format_stamp: "Formatted at 2026-08-11 07:38:46 on dist-test-slave-zkpd"
server_key: "da1d43dfa3b18073d6fc2a1e6fbf5f0d"
server_key_iv: "33aaaf81b96d545dd08082035ddb486c"
server_key_version: "encryptionkey@0"
I20260811 07:38:46.890708 19913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:46.912714 19913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:46.915993 19913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:46.916194 19913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:46.924836 19913 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:44301
I20260811 07:38:46.924863 19995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:44301 every 8 connection(s)
I20260811 07:38:46.925792 19913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:38:46.926972 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 19913
I20260811 07:38:46.927109 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:38:46.927362 18723 external_mini_cluster.cc:1428] Setting key f03769f5899baa59fcd6003445957527
I20260811 07:38:46.928911 19996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:46.934525 19996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0: Bootstrap starting.
Aug 11 07:38:46 dist-test-slave-zkpd krb5kdc[19899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:46.937134 19996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:46.938228 19996 log.cc:824] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:46.940529 19996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0: No bootstrap required, opened a new log
I20260811 07:38:46.941761 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:46.928821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.18.72.254:44301)
0811 07:38:46.929268 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:46.929299 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:46.929337 (+ 38us) server_negotiation.cc:407] Connection header received
0811 07:38:46.929982 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:46.929998 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:46.930338 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:46.930671 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:46.931488 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:46.932564 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:46.933416 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:46.933896 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:46.934080 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:46.936952 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:46.936958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:46.936971 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:46.937000 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:46.938909 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:46.939421 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:46.939425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:46.939430 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:46.939536 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:46.939829 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:46.939831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:46.939832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:46.940262 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:46.940407 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:46.940751 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 07:38:46.941071 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":136,"threads_started":1}
I20260811 07:38:46.943967 19996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c317794dd84712a50b2d18b38711e0" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44301 } }
I20260811 07:38:46.944207 19996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:46.944286 19996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98c317794dd84712a50b2d18b38711e0, State: Initialized, Role: FOLLOWER
I20260811 07:38:46.944722 19996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c317794dd84712a50b2d18b38711e0" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44301 } }
I20260811 07:38:46.944862 19996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:38:46.944928 19996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:38:46.945043 19996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:46.945925 19996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c317794dd84712a50b2d18b38711e0" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44301 } }
I20260811 07:38:46.946264 19996 leader_election.cc:302] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98c317794dd84712a50b2d18b38711e0; no voters:
I20260811 07:38:46.946568 19996 leader_election.cc:288] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:38:46.946705 20001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:46.946939 20001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [term 1 LEADER]: Becoming Leader. State: Replica: 98c317794dd84712a50b2d18b38711e0, State: Running, Role: LEADER
I20260811 07:38:46.947235 20001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c317794dd84712a50b2d18b38711e0" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44301 } }
I20260811 07:38:46.947779 19996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:38:46.949067 20002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98c317794dd84712a50b2d18b38711e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c317794dd84712a50b2d18b38711e0" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44301 } } }
I20260811 07:38:46.949112 20003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98c317794dd84712a50b2d18b38711e0. Latest consensus state: current_term: 1 leader_uuid: "98c317794dd84712a50b2d18b38711e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c317794dd84712a50b2d18b38711e0" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44301 } } }
I20260811 07:38:46.949198 20002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:46.949221 20003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0 [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:46.949573 20011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:38:46.952368 20011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:38:46.958135 20011 catalog_manager.cc:1347] Generated new cluster ID: ae5e794b58274afdb8206712b23c5b9f
I20260811 07:38:46.958216 20011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:38:46.968271 20011 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:38:46.969151 20011 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:38:46.972764 20011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0: Generated new TSK 0
I20260811 07:38:46.973479 20011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:38:47.011230 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44301
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:45999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:47.154426 20027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:47.154662 20027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:47.154704 20027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:47.158618 20027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:47.158675 20027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:47.158748 20027 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:38:47.158812 20027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:38:47.163947 20027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:45999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:47.164951 20027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:47.165967 20027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:47.175436 20033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:47.175463 20034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:47.175561 20027 server_base.cc:1034] running on GCE node
W20260811 07:38:47.175437 20036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:47.176138 20027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:47.176708 20027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:47.177868 20027 hybrid_clock.cc:648] HybridClock initialized: now 1786433927177853 us; error 42 us; skew 500 ppm
Aug 11 07:38:47 dist-test-slave-zkpd krb5kdc[19899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:47.180619 20027 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:38:47.181438 20027 webserver.cc:460] Webserver started at http://127.18.72.193:40701/ using document root <none> and password file <none>
I20260811 07:38:47.181986 20027 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:47.182047 20027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:47.182294 20027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:47.184407 20027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "a96bbdc97e0347bfb88bcfc65e00467c"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "0bdbec60321de815b0e09b723851f7de"
server_key_iv: "e8921c730f3bafd4f9b9bb5083d76dd9"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.184968 20027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "a96bbdc97e0347bfb88bcfc65e00467c"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "0bdbec60321de815b0e09b723851f7de"
server_key_iv: "e8921c730f3bafd4f9b9bb5083d76dd9"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.188633 20027 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:38:47.190958 20043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:47.192205 20027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 07:38:47.192341 20027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "a96bbdc97e0347bfb88bcfc65e00467c"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "0bdbec60321de815b0e09b723851f7de"
server_key_iv: "e8921c730f3bafd4f9b9bb5083d76dd9"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.192451 20027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:47.204663 20027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:47.207307 20027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:47.207505 20027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:47.208978 20027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:47.209055 20027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:47.209125 20027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:47.209175 20027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:38:47 dist-test-slave-zkpd krb5kdc[19899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:47.220463 20027 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:39441
I20260811 07:38:47.220525 20168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:39441 every 8 connection(s)
I20260811 07:38:47.221571 20027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:38:47.223991 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.209816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:52473 (local address 127.18.72.254:44301)
0811 07:38:47.209953 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.209958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.211025 (+ 1067us) server_negotiation.cc:407] Connection header received
0811 07:38:47.212151 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.212154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.212223 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.212284 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:47.214168 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.214799 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.216697 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.217096 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.217181 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.220127 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:47.220133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:47.220137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:47.220170 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:47.221781 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.222396 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.222400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.222404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.222499 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.222863 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.222866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.222868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.223057 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:47.223162 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.223654 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.223786 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:38:47.224736 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.210343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44301 (local address 127.18.72.193:52473)
0811 07:38:47.210881 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.210924 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.211858 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.212790 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.212801 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.213305 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:47.214009 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.214033 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.214958 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.214962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.216560 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.216573 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.217224 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.217229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.217425 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.218020 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:47.218042 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:47.219963 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:47.221901 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.221907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.221921 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.222273 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.222596 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.222602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.222604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.222747 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.223174 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:47.223181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:47.223418 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.223592 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.223897 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":136,"threads_started":1}
I20260811 07:38:47.226002 20169 heartbeater.cc:347] Connected to a master server at 127.18.72.254:44301
I20260811 07:38:47.226161 20169 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:47.226689 20169 heartbeater.cc:511] Master 127.18.72.254:44301 requested a full tablet report, sending...
I20260811 07:38:47.228448 19945 ts_manager.cc:194] Registered new tserver with Master: a96bbdc97e0347bfb88bcfc65e00467c (127.18.72.193:39441)
I20260811 07:38:47.229732 19945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:52473
I20260811 07:38:47.231226 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 20027
I20260811 07:38:47.231348 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:38:47.231652 18723 external_mini_cluster.cc:1428] Setting key 21f1c64a1837c23f9acab158127bddf4
I20260811 07:38:47.235529 20177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.228094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44301 (local address 127.18.72.193:35035)
0811 07:38:47.228383 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.228399 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.228496 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.228780 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.228783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.228980 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:47.229094 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.229100 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.229824 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.229827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.230652 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.230661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.231197 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.231203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.231312 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.231824 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:47.231839 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:47.232245 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:47.233979 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.233985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.233990 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.234261 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.234521 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.234527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.234532 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.234635 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.234975 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:47.234981 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:47.235093 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.235212 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.235384 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":117,"threads_started":1}
I20260811 07:38:47.235626 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.228166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:35035 (local address 127.18.72.254:44301)
0811 07:38:47.228302 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.228306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.228468 (+ 162us) server_negotiation.cc:407] Connection header received
0811 07:38:47.228617 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.228622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.228671 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.228735 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:47.229204 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.229695 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.230770 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.231070 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.231179 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.232353 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:47.232357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:47.232359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:47.232383 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:47.233868 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.234358 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.234360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.234361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.234430 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.234725 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.234727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.234728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.234884 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:47.234958 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.235283 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.235425 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:38:47.258955 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44301
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:45999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:47.373375 20180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:47.373633 20180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:47.373730 20180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:47.378121 20180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:47.378175 20180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:47.378218 20180 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:38:47.378248 20180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:38:47.383275 20180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:45999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:47.384263 20180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:47.385259 20180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:47.395006 20188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:47.394946 20186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:47.394927 20185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:47.395246 20180 server_base.cc:1034] running on GCE node
I20260811 07:38:47.395742 20180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:47.396482 20180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:47.397676 20180 hybrid_clock.cc:648] HybridClock initialized: now 1786433927397647 us; error 42 us; skew 500 ppm
Aug 11 07:38:47 dist-test-slave-zkpd krb5kdc[19899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:47.404675 20180 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:38:47.405573 20180 webserver.cc:460] Webserver started at http://127.18.72.194:43237/ using document root <none> and password file <none>
I20260811 07:38:47.406178 20180 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:47.406271 20180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:47.406513 20180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:47.408682 20180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "c381f1ff26854195a6a1d5703cd086ca"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "dc37c3038ca5bbbae916595b1b2ecb58"
server_key_iv: "147ed7cd6a98f3e9122e13b98e2e1619"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.409260 20180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "c381f1ff26854195a6a1d5703cd086ca"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "dc37c3038ca5bbbae916595b1b2ecb58"
server_key_iv: "147ed7cd6a98f3e9122e13b98e2e1619"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.412979 20180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:38:47.415355 20197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:47.416559 20180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:38:47.416693 20180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "c381f1ff26854195a6a1d5703cd086ca"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "dc37c3038ca5bbbae916595b1b2ecb58"
server_key_iv: "147ed7cd6a98f3e9122e13b98e2e1619"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.416815 20180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:47.426908 20180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:47.429558 20180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:47.429747 20180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:47.431098 20180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:47.431169 20180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:47.431236 20180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:47.431272 20180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:38:47 dist-test-slave-zkpd krb5kdc[19899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:47.442654 20180 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:33681
I20260811 07:38:47.442936 20322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:33681 every 8 connection(s)
I20260811 07:38:47.443765 20180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:38:47.445399 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 20180
I20260811 07:38:47.445508 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:38:47.445732 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.431750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:56311 (local address 127.18.72.254:44301)
0811 07:38:47.431873 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.431877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.433504 (+ 1627us) server_negotiation.cc:407] Connection header received
0811 07:38:47.434708 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.434712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.434773 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.434878 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:47.436199 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.436750 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.437874 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.438115 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.438225 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.441851 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:47.441856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:47.441858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:47.441886 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:47.443580 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.444167 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.444171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.444173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.444273 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.444698 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.444700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.444702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.444881 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:47.444971 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.445445 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.445576 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:38:47.445776 18723 external_mini_cluster.cc:1428] Setting key f61de929a68f9190c33c73713104e172
I20260811 07:38:47.446563 20207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.432846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44301 (local address 127.18.72.194:56311)
0811 07:38:47.433351 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.433406 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.434500 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.435072 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.435083 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.435466 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:47.436060 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.436076 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.436897 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.436901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.437745 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.437755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.438290 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.438295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.438474 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.439051 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:47.439075 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:47.441677 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:47.443686 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.443691 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.443702 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.444046 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.444399 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.444404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.444409 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.444584 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.444987 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:47.444991 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:47.445191 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.445337 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.445635 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":128,"threads_started":1}
I20260811 07:38:47.447760 20323 heartbeater.cc:347] Connected to a master server at 127.18.72.254:44301
I20260811 07:38:47.448000 20323 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:47.448508 20323 heartbeater.cc:511] Master 127.18.72.254:44301 requested a full tablet report, sending...
I20260811 07:38:47.449703 19945 ts_manager.cc:194] Registered new tserver with Master: c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681)
I20260811 07:38:47.450335 19945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:56311
I20260811 07:38:47.458604 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.450311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44301 (local address 127.18.72.194:58287)
0811 07:38:47.450596 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.450613 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.450749 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.451112 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.451116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.451359 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:47.451542 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.451552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.452475 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.452481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.453461 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.453478 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.453935 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.453939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.454064 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.454520 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:47.454539 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:47.454984 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:47.456966 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.456969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.456970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.457151 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.457455 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.457457 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.457459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.457530 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.458031 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:47.458033 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:47.458135 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.458254 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.458388 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260811 07:38:47.458743 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.450373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:58287 (local address 127.18.72.254:44301)
0811 07:38:47.450496 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.450500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.450679 (+ 179us) server_negotiation.cc:407] Connection header received
0811 07:38:47.450855 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.450860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.450909 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.450961 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:47.451684 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.452347 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.453602 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.453814 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.453888 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.455143 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:47.455150 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:47.455157 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:47.455188 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:47.456858 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.457248 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.457252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.457254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.457357 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.457623 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.457626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.457628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.457919 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:47.458040 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.458381 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.458541 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:38:47.478591 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44301
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:45999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:47.590487 20332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:47.590764 20332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:47.590811 20332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:47.594873 20332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:47.594934 20332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:47.594973 20332 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:38:47.595005 20332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:38:47.600375 20332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:45999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:47.601344 20332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:47.602301 20332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:47.611701 20338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:47.611702 20337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:47.611755 20341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:47.612026 20332 server_base.cc:1034] running on GCE node
I20260811 07:38:47.612460 20332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:47.613116 20332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:47.614303 20332 hybrid_clock.cc:648] HybridClock initialized: now 1786433927614287 us; error 25 us; skew 500 ppm
Aug 11 07:38:47 dist-test-slave-zkpd krb5kdc[19899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:47.617033 20332 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:38:47.618172 20332 webserver.cc:460] Webserver started at http://127.18.72.195:38355/ using document root <none> and password file <none>
I20260811 07:38:47.618921 20332 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:47.619017 20332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:47.619272 20332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:47.622198 20332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "e4e4906ca1ad49f9969a904b34950acf"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "4fb8c65078018c228c3ecca16fddbcb4"
server_key_iv: "8b1457b26196fd8b0808071562a57829"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.622947 20332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "e4e4906ca1ad49f9969a904b34950acf"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "4fb8c65078018c228c3ecca16fddbcb4"
server_key_iv: "8b1457b26196fd8b0808071562a57829"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.628364 20332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 07:38:47.631388 20349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:47.632655 20332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:38:47.632783 20332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "e4e4906ca1ad49f9969a904b34950acf"
format_stamp: "Formatted at 2026-08-11 07:38:47 on dist-test-slave-zkpd"
server_key: "4fb8c65078018c228c3ecca16fddbcb4"
server_key_iv: "8b1457b26196fd8b0808071562a57829"
server_key_version: "encryptionkey@0"
I20260811 07:38:47.632892 20332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:47.641587 20332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:47.644420 20332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:47.644611 20332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:47.646085 20332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:47.646132 20332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:47.646171 20332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:47.646220 20332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:38:47 dist-test-slave-zkpd krb5kdc[19899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:47.657531 20332 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:42019
I20260811 07:38:47.657660 20479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:42019 every 8 connection(s)
I20260811 07:38:47.658557 20332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:38:47.660607 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.646771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:50247 (local address 127.18.72.254:44301)
0811 07:38:47.646944 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.646949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.648363 (+ 1414us) server_negotiation.cc:407] Connection header received
0811 07:38:47.649757 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.649760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.649833 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.649929 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:47.651655 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.652564 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.653782 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.654016 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.654135 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.656853 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:47.656856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:47.656858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:47.656883 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:47.658446 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.659085 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.659088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.659090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.659188 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.659549 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.659553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.659555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.659750 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:47.659835 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.660391 (+ 556us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.660493 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 07:38:47.661415 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.647660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44301 (local address 127.18.72.195:50247)
0811 07:38:47.648209 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.648262 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.649508 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.650166 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.650178 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.650713 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:47.651470 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.651494 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.652686 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.652690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.653624 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.653634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.654174 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.654179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.654366 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.654969 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:47.654985 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:47.656676 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:47.658565 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.658574 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.658591 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.658955 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.659300 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.659306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.659309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.659447 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.659856 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:47.659861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:47.660176 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.660340 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.660582 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":187,"threads_started":1}
I20260811 07:38:47.662566 20480 heartbeater.cc:347] Connected to a master server at 127.18.72.254:44301
I20260811 07:38:47.662707 20480 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:47.663276 20480 heartbeater.cc:511] Master 127.18.72.254:44301 requested a full tablet report, sending...
I20260811 07:38:47.664419 19945 ts_manager.cc:194] Registered new tserver with Master: e4e4906ca1ad49f9969a904b34950acf (127.18.72.195:42019)
I20260811 07:38:47.664987 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 20332
I20260811 07:38:47.665124 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:38:47.665243 19945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:50247
I20260811 07:38:47.665441 18723 external_mini_cluster.cc:1428] Setting key 6592ec7a522ba608a614e68b45f7969e
I20260811 07:38:47.667549 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:38:47.673336 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.664973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:47115 (local address 127.18.72.254:44301)
0811 07:38:47.665092 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.665097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.665973 (+ 876us) server_negotiation.cc:407] Connection header received
0811 07:38:47.666121 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.666124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.666163 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.666255 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:47.666921 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.667413 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.668651 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.669057 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.669170 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.670277 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:47.670281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:47.670283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:47.670311 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:47.671633 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.672168 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.672171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.672172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.672249 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.672569 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.672578 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.672580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.672760 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:47.672843 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.673136 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.673224 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:38:47.673662 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.664886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44301 (local address 127.18.72.195:47115)
0811 07:38:47.665892 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.665907 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.666007 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.666271 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.666273 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.666667 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:47.666811 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.666818 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.668001 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.668004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.668535 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.668542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.669181 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.669184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.669251 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.669779 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:47.669799 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:47.670145 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:47.671747 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.671753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.671758 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.672051 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.672346 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:47.672352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:47.672357 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:47.672459 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:47.672861 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:47.672867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:47.672979 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.673097 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.673257 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"thread_start_us":103,"threads_started":1}
Aug 11 07:38:47 dist-test-slave-zkpd krb5kdc[19899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:38:47 dist-test-slave-zkpd krb5kdc[19899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:47.696175 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.688077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38306 (local address 127.18.72.254:44301)
0811 07:38:47.688201 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.688204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.688216 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:38:47.688254 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.688256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.688294 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.688394 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:47.689099 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.689618 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.690433 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.690710 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.690824 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.693190 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:47.693194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:47.693197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:47.693226 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:47.694541 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.694944 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.694948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.694950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.695030 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:47.695306 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:47.695312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:47.695314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:47.695502 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:47.695638 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.695879 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.696012 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:38:47.698963 19945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:38:47.701970 19945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:47.716240 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.711040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.18.72.194:33681)
0811 07:38:47.711329 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.711341 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.711366 (+ 25us) server_negotiation.cc:407] Connection header received
0811 07:38:47.711445 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.711452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.711661 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.712017 (+ 356us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:47.712377 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.713597 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.714994 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.715562 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.715690 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.715873 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.715996 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.716085 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
I20260811 07:38:47.716238 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.710165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:33681 (local address 127.0.0.1:36966)
0811 07:38:47.710796 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.710809 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.710902 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.711850 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.711856 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.711932 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:47.712234 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.712239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.713732 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.713736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.714877 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.714887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.715688 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.715691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.715798 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.715833 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.715961 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.716082 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":218,"spinlock_wait_cycles":30080,"thread_start_us":122,"threads_started":1}
I20260811 07:38:47.716316 20498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.710091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39441 (local address 127.0.0.1:52302)
0811 07:38:47.710421 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.710480 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.710610 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.711850 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.711856 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.711932 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:47.712077 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.712084 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.713395 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.713401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.714782 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.714793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.715869 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.715873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.715983 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.716007 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.716066 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.716137 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":53,"spinlock_wait_cycles":6272,"thread_start_us":71,"threads_started":1}
I20260811 07:38:47.716934 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.711107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.18.72.195:42019)
0811 07:38:47.711413 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.711425 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.711849 (+ 424us) server_negotiation.cc:407] Connection header received
0811 07:38:47.711969 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.711974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.712183 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.712331 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:47.712532 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.713599 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.714961 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.715724 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.715850 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.716026 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.716746 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.716829 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":119,"threads_started":1}
I20260811 07:38:47.717080 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.710627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.18.72.193:39441)
0811 07:38:47.711438 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.711451 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.711470 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:38:47.711532 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.711536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.711675 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.712434 (+ 759us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:47.712585 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.713277 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.714930 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.715732 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.716667 (+ 935us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.716818 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.716866 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.716950 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":129,"threads_started":1}
I20260811 07:38:47.717249 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.710812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42019 (local address 127.0.0.1:41216)
0811 07:38:47.711637 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.711650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.711740 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.712311 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.712317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.712337 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:47.712423 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.712429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.713770 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.713783 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.714822 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.714832 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.716083 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.716087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.716160 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.716181 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.716244 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.717109 (+ 865us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":7,"thread_start_us":91,"threads_started":1}
I20260811 07:38:47.719149 20249 tablet_service.cc:1467] Processing CreateTablet for tablet f4a8e18844114af2a3b338a88111e5af (DEFAULT_TABLE table=test-table [id=e27f1a92bb214be2ac9553fddff72f17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:47.719766 20407 tablet_service.cc:1467] Processing CreateTablet for tablet f4a8e18844114af2a3b338a88111e5af (DEFAULT_TABLE table=test-table [id=e27f1a92bb214be2ac9553fddff72f17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:47.719766 20095 tablet_service.cc:1467] Processing CreateTablet for tablet f4a8e18844114af2a3b338a88111e5af (DEFAULT_TABLE table=test-table [id=e27f1a92bb214be2ac9553fddff72f17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:47.720335 20249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a8e18844114af2a3b338a88111e5af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:47.720880 20095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a8e18844114af2a3b338a88111e5af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:47.720902 20407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a8e18844114af2a3b338a88111e5af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:47.727507 20504 tablet_bootstrap.cc:492] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: Bootstrap starting.
I20260811 07:38:47.729120 20505 tablet_bootstrap.cc:492] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: Bootstrap starting.
I20260811 07:38:47.729468 20506 tablet_bootstrap.cc:492] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: Bootstrap starting.
I20260811 07:38:47.730163 20504 tablet_bootstrap.cc:654] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:47.730872 20504 log.cc:824] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:47.731144 20505 tablet_bootstrap.cc:654] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:47.731240 20506 tablet_bootstrap.cc:654] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:47.732017 20505 log.cc:824] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:47.732017 20506 log.cc:824] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:47.732888 20504 tablet_bootstrap.cc:492] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: No bootstrap required, opened a new log
I20260811 07:38:47.733094 20504 ts_tablet_manager.cc:1397] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 07:38:47.733929 20505 tablet_bootstrap.cc:492] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: No bootstrap required, opened a new log
I20260811 07:38:47.733929 20506 tablet_bootstrap.cc:492] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: No bootstrap required, opened a new log
I20260811 07:38:47.734108 20505 ts_tablet_manager.cc:1397] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 07:38:47.734874 20506 ts_tablet_manager.cc:1397] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 07:38:47.736914 20504 raft_consensus.cc:348] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.737213 20504 raft_consensus.cc:374] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:47.737277 20504 raft_consensus.cc:729] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c381f1ff26854195a6a1d5703cd086ca, State: Initialized, Role: FOLLOWER
I20260811 07:38:47.737807 20504 consensus_queue.cc:260] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.738241 20505 raft_consensus.cc:348] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.738528 20505 raft_consensus.cc:374] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:47.738592 20505 raft_consensus.cc:729] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96bbdc97e0347bfb88bcfc65e00467c, State: Initialized, Role: FOLLOWER
I20260811 07:38:47.738868 20323 heartbeater.cc:503] Master 127.18.72.254:44301 was elected leader, sending a full tablet report...
I20260811 07:38:47.738885 20504 ts_tablet_manager.cc:1428] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:38:47.739056 20505 consensus_queue.cc:260] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.739665 20169 heartbeater.cc:503] Master 127.18.72.254:44301 was elected leader, sending a full tablet report...
I20260811 07:38:47.740427 20506 raft_consensus.cc:348] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.740664 20506 raft_consensus.cc:374] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:47.740721 20506 raft_consensus.cc:729] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e4906ca1ad49f9969a904b34950acf, State: Initialized, Role: FOLLOWER
I20260811 07:38:47.741000 20505 ts_tablet_manager.cc:1428] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 07:38:47.741146 20506 consensus_queue.cc:260] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.741742 20480 heartbeater.cc:503] Master 127.18.72.254:44301 was elected leader, sending a full tablet report...
I20260811 07:38:47.742043 20506 ts_tablet_manager.cc:1428] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260811 07:38:47.840576 20512 raft_consensus.cc:482] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:47.840840 20512 raft_consensus.cc:504] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.842088 20512 leader_election.cc:288] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681), e4e4906ca1ad49f9969a904b34950acf (127.18.72.195:42019)
I20260811 07:38:47.845992 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.842480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:59293 (local address 127.18.72.194:33681)
0811 07:38:47.842616 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.842620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.842657 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:38:47.842856 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.842859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.842911 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.843009 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:47.843390 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.843974 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.845050 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.845442 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.845569 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.845624 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.845805 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.845871 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:38:47.846009 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.842319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:33681 (local address 127.18.72.193:59293)
0811 07:38:47.842584 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.842601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.842716 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.843127 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.843130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.843150 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:47.843265 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.843270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.844083 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.844088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.844921 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.844931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.845561 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.845565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.845676 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.845696 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.845788 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.845856 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260811 07:38:47.846084 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.842904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:58341 (local address 127.18.72.195:42019)
0811 07:38:47.843030 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.843034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.843049 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:38:47.843185 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.843188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.843239 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.843325 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:47.843587 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.844110 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.845120 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.845524 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.845653 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.845706 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.845895 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.845962 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:38:47.846127 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.842738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42019 (local address 127.18.72.193:58341)
0811 07:38:47.842990 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:47.843001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:47.843069 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:47.843340 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:47.843343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:47.843358 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:47.843478 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.843483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.844225 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.844230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.844991 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:47.845000 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.845649 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.845652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:47.845768 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.845786 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 07:38:47.845880 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 07:38:47.845949 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
I20260811 07:38:47.846808 20429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a8e18844114af2a3b338a88111e5af" candidate_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4e4906ca1ad49f9969a904b34950acf" is_pre_election: true
I20260811 07:38:47.846836 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a8e18844114af2a3b338a88111e5af" candidate_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c381f1ff26854195a6a1d5703cd086ca" is_pre_election: true
I20260811 07:38:47.847123 20272 raft_consensus.cc:2454] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a96bbdc97e0347bfb88bcfc65e00467c in term 0.
I20260811 07:38:47.847127 20429 raft_consensus.cc:2454] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a96bbdc97e0347bfb88bcfc65e00467c in term 0.
I20260811 07:38:47.847633 20044 leader_election.cc:302] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96bbdc97e0347bfb88bcfc65e00467c, c381f1ff26854195a6a1d5703cd086ca; no voters:
I20260811 07:38:47.848176 20512 raft_consensus.cc:2781] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:47.848295 20512 raft_consensus.cc:482] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:47.848363 20512 raft_consensus.cc:3037] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:47.849624 20512 raft_consensus.cc:504] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.849977 20512 leader_election.cc:288] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [CANDIDATE]: Term 1 election: Requested vote from peers c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681), e4e4906ca1ad49f9969a904b34950acf (127.18.72.195:42019)
I20260811 07:38:47.850353 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a8e18844114af2a3b338a88111e5af" candidate_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c381f1ff26854195a6a1d5703cd086ca"
I20260811 07:38:47.850472 20429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a8e18844114af2a3b338a88111e5af" candidate_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4e4906ca1ad49f9969a904b34950acf"
I20260811 07:38:47.850492 20272 raft_consensus.cc:3037] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:47.850612 20429 raft_consensus.cc:3037] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:47.851974 20272 raft_consensus.cc:2454] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a96bbdc97e0347bfb88bcfc65e00467c in term 1.
I20260811 07:38:47.851986 20429 raft_consensus.cc:2454] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a96bbdc97e0347bfb88bcfc65e00467c in term 1.
I20260811 07:38:47.852317 20044 leader_election.cc:302] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96bbdc97e0347bfb88bcfc65e00467c, e4e4906ca1ad49f9969a904b34950acf; no voters:
I20260811 07:38:47.852522 20512 raft_consensus.cc:2781] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:47.852784 20512 raft_consensus.cc:686] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 1 LEADER]: Becoming Leader. State: Replica: a96bbdc97e0347bfb88bcfc65e00467c, State: Running, Role: LEADER
I20260811 07:38:47.853155 20512 consensus_queue.cc:237] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.856410 19946 catalog_manager.cc:5495] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c reported cstate change: term changed from 0 to 1, leader changed from <none> to a96bbdc97e0347bfb88bcfc65e00467c (127.18.72.193). New cstate: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } health_report { overall_health: UNKNOWN } } }
I20260811 07:38:47.879537 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:47.875681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.18.72.193:39441)
0811 07:38:47.875823 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:47.875826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:47.875845 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:38:47.876127 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:47.876130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:47.876186 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:47.876293 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:47.876584 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.877101 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.878004 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:47.878227 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:47.878345 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:47.878685 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:47.879078 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:47.879231 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:47.879328 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 07:38:47.879388 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:38:47.886287 20272 raft_consensus.cc:1261] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Refusing update from remote peer a96bbdc97e0347bfb88bcfc65e00467c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:47.886300 20429 raft_consensus.cc:1261] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Refusing update from remote peer a96bbdc97e0347bfb88bcfc65e00467c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:47.887032 20512 consensus_queue.cc:1035] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:47.887569 20516 consensus_queue.cc:1035] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:47.900354 20521 mvcc.cc:205] Tried to move back new op lower bound from 7317233368613867520 to 7317233368489381888. Current Snapshot: MvccSnapshot[applied={T|T < 7317233368613867520}]
I20260811 07:38:47.900379 20519 mvcc.cc:205] Tried to move back new op lower bound from 7317233368613867520 to 7317233368489381888. Current Snapshot: MvccSnapshot[applied={T|T < 7317233368613867520 or (T in {7317233368613867520})}]
I20260811 07:38:47.902199 20520 mvcc.cc:205] Tried to move back new op lower bound from 7317233368613867520 to 7317233368489381888. Current Snapshot: MvccSnapshot[applied={T|T < 7317233368613867520 or (T in {7317233368613867520})}]
I20260811 07:38:47.908613 19946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:47.908809 19946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:47.911553 19946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0: Sending DeleteTablet for 3 replicas of tablet f4a8e18844114af2a3b338a88111e5af
I20260811 07:38:47.912431 20095 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a8e18844114af2a3b338a88111e5af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:52302
I20260811 07:38:47.912541 20249 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a8e18844114af2a3b338a88111e5af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:36966
I20260811 07:38:47.912642 20407 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a8e18844114af2a3b338a88111e5af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:41216
I20260811 07:38:47.913070 20532 tablet_replica.cc:330] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: stopping tablet replica
I20260811 07:38:47.913017 19946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 07:38:47.913375 20532 raft_consensus.cc:2229] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 07:38:47.913419 19946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:47.913707 20532 raft_consensus.cc:2258] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:47.915091 20532 ts_tablet_manager.cc:1905] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:47.917834 20532 ts_tablet_manager.cc:1918] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:38:47.917930 20532 log.cc:1196] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/f4a8e18844114af2a3b338a88111e5af
I20260811 07:38:47.918229 20532 ts_tablet_manager.cc:1939] T f4a8e18844114af2a3b338a88111e5af P e4e4906ca1ad49f9969a904b34950acf: Deleting consensus metadata
I20260811 07:38:47.919651 20407 tablet_service.cc:1467] Processing CreateTablet for tablet 359c499e71a64120b8c24527127c86f7 (DEFAULT_TABLE table=test-table [id=f8636902f42b4864b1ac1d25e736e2c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:47.919848 20095 tablet_service.cc:1467] Processing CreateTablet for tablet 359c499e71a64120b8c24527127c86f7 (DEFAULT_TABLE table=test-table [id=f8636902f42b4864b1ac1d25e736e2c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:47.919934 20407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359c499e71a64120b8c24527127c86f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:47.919967 20249 tablet_service.cc:1467] Processing CreateTablet for tablet 359c499e71a64120b8c24527127c86f7 (DEFAULT_TABLE table=test-table [id=f8636902f42b4864b1ac1d25e736e2c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:47.920114 20095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359c499e71a64120b8c24527127c86f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:47.920288 20249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359c499e71a64120b8c24527127c86f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:47.921712 20531 tablet_replica.cc:330] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: stopping tablet replica
I20260811 07:38:47.922011 20531 raft_consensus.cc:2229] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:47.922293 20531 raft_consensus.cc:2258] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:47.922434 20506 tablet_bootstrap.cc:492] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: Bootstrap starting.
I20260811 07:38:47.922667 20505 tablet_bootstrap.cc:492] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: Bootstrap starting.
I20260811 07:38:47.923262 20504 tablet_bootstrap.cc:492] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: Bootstrap starting.
I20260811 07:38:47.923393 20506 tablet_bootstrap.cc:654] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:47.923676 20505 tablet_bootstrap.cc:654] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:47.924538 20504 tablet_bootstrap.cc:654] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:47.924510 19930 catalog_manager.cc:4841] TS e4e4906ca1ad49f9969a904b34950acf (127.18.72.195:42019): tablet f4a8e18844114af2a3b338a88111e5af (table test-table [id=e27f1a92bb214be2ac9553fddff72f17]) successfully deleted
I20260811 07:38:47.924613 20529 tablet_replica.cc:330] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: stopping tablet replica
I20260811 07:38:47.924901 20529 raft_consensus.cc:2229] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:38:47.925345 20529 raft_consensus.cc:2258] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:47.925392 20506 tablet_bootstrap.cc:492] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: No bootstrap required, opened a new log
I20260811 07:38:47.925459 20506 ts_tablet_manager.cc:1397] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 07:38:47.925885 20506 raft_consensus.cc:348] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.926002 20506 raft_consensus.cc:374] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:47.926039 20506 raft_consensus.cc:729] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e4906ca1ad49f9969a904b34950acf, State: Initialized, Role: FOLLOWER
I20260811 07:38:47.926151 20506 consensus_queue.cc:260] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.926422 20506 ts_tablet_manager.cc:1428] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 07:38:47.927095 20504 tablet_bootstrap.cc:492] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: No bootstrap required, opened a new log
I20260811 07:38:47.927196 20504 ts_tablet_manager.cc:1397] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 07:38:47.927640 20504 raft_consensus.cc:348] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.927738 20504 raft_consensus.cc:374] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:47.927769 20504 raft_consensus.cc:729] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c381f1ff26854195a6a1d5703cd086ca, State: Initialized, Role: FOLLOWER
I20260811 07:38:47.927912 20504 consensus_queue.cc:260] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.928143 20504 ts_tablet_manager.cc:1428] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:38:47.928182 20531 ts_tablet_manager.cc:1905] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:47.929648 20529 ts_tablet_manager.cc:1905] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:47.931439 20531 ts_tablet_manager.cc:1918] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:38:47.931538 20531 log.cc:1196] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/f4a8e18844114af2a3b338a88111e5af
I20260811 07:38:47.932088 20529 ts_tablet_manager.cc:1918] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:38:47.932174 20529 log.cc:1196] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/f4a8e18844114af2a3b338a88111e5af
I20260811 07:38:47.932389 20531 ts_tablet_manager.cc:1939] T f4a8e18844114af2a3b338a88111e5af P c381f1ff26854195a6a1d5703cd086ca: Deleting consensus metadata
I20260811 07:38:47.932472 20529 ts_tablet_manager.cc:1939] T f4a8e18844114af2a3b338a88111e5af P a96bbdc97e0347bfb88bcfc65e00467c: Deleting consensus metadata
I20260811 07:38:47.933426 19930 catalog_manager.cc:4841] TS c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681): tablet f4a8e18844114af2a3b338a88111e5af (table test-table [id=e27f1a92bb214be2ac9553fddff72f17]) successfully deleted
I20260811 07:38:47.933451 19933 catalog_manager.cc:4841] TS a96bbdc97e0347bfb88bcfc65e00467c (127.18.72.193:39441): tablet f4a8e18844114af2a3b338a88111e5af (table test-table [id=e27f1a92bb214be2ac9553fddff72f17]) successfully deleted
I20260811 07:38:47.934302 20505 tablet_bootstrap.cc:492] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: No bootstrap required, opened a new log
I20260811 07:38:47.934401 20505 ts_tablet_manager.cc:1397] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20260811 07:38:47.934964 20505 raft_consensus.cc:348] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.935108 20505 raft_consensus.cc:374] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:47.935159 20505 raft_consensus.cc:729] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96bbdc97e0347bfb88bcfc65e00467c, State: Initialized, Role: FOLLOWER
I20260811 07:38:47.935309 20505 consensus_queue.cc:260] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:47.935650 20505 ts_tablet_manager.cc:1428] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260811 07:38:47.944933 20324 tablet.cc:2369] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:38:47.972981 20171 tablet.cc:2369] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:38:48.072602 20513 raft_consensus.cc:482] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:48.072775 20513 raft_consensus.cc:504] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.073959 20513 leader_election.cc:288] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a96bbdc97e0347bfb88bcfc65e00467c (127.18.72.193:39441), c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681)
I20260811 07:38:48.078316 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.074216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:33681 (local address 127.18.72.195:33543)
0811 07:38:48.074548 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:48.074563 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:48.074665 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:48.075027 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:48.075030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:48.075048 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:48.075228 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.075234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.076233 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.076236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.077071 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.077080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.077860 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.077864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.077974 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.078001 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 07:38:48.078073 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:38:48.078141 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":43,"thread_start_us":70,"threads_started":1}
I20260811 07:38:48.078342 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.074662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:33543 (local address 127.18.72.194:33681)
0811 07:38:48.074800 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.074804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.074818 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:38:48.074873 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.074878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.074922 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.074999 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:48.075349 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.076104 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.077216 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.077715 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.077851 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.077904 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.078117 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.078199 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 07:38:48.078804 20360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.074144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39441 (local address 127.18.72.195:36055)
0811 07:38:48.074366 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:48.074382 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:48.074480 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:48.074908 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:48.074912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:48.074930 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:48.075091 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.075099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.076207 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.076211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.077425 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.077436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.078387 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.078390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.078506 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.078534 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 07:38:48.078607 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:38:48.078680 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":2}
I20260811 07:38:48.078778 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359c499e71a64120b8c24527127c86f7" candidate_uuid: "e4e4906ca1ad49f9969a904b34950acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c381f1ff26854195a6a1d5703cd086ca" is_pre_election: true
I20260811 07:38:48.078931 20272 raft_consensus.cc:2454] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4e4906ca1ad49f9969a904b34950acf in term 0.
I20260811 07:38:48.079211 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.074445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:36055 (local address 127.18.72.193:39441)
0811 07:38:48.074576 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.074580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.074596 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:38:48.074650 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.074653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.074702 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.074774 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:48.075239 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.076083 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.077564 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.078160 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.078275 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.078323 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.079018 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.079099 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:38:48.079349 20351 leader_election.cc:302] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c381f1ff26854195a6a1d5703cd086ca, e4e4906ca1ad49f9969a904b34950acf; no voters:
I20260811 07:38:48.079612 20513 raft_consensus.cc:2781] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:48.079701 20513 raft_consensus.cc:482] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:48.079754 20513 raft_consensus.cc:3037] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:48.079694 20118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359c499e71a64120b8c24527127c86f7" candidate_uuid: "e4e4906ca1ad49f9969a904b34950acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" is_pre_election: true
I20260811 07:38:48.079952 20118 raft_consensus.cc:2454] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4e4906ca1ad49f9969a904b34950acf in term 0.
I20260811 07:38:48.081100 20513 raft_consensus.cc:504] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.081483 20513 leader_election.cc:288] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [CANDIDATE]: Term 1 election: Requested vote from peers a96bbdc97e0347bfb88bcfc65e00467c (127.18.72.193:39441), c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681)
I20260811 07:38:48.081790 20118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359c499e71a64120b8c24527127c86f7" candidate_uuid: "e4e4906ca1ad49f9969a904b34950acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96bbdc97e0347bfb88bcfc65e00467c"
I20260811 07:38:48.081791 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359c499e71a64120b8c24527127c86f7" candidate_uuid: "e4e4906ca1ad49f9969a904b34950acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c381f1ff26854195a6a1d5703cd086ca"
I20260811 07:38:48.081910 20118 raft_consensus.cc:3037] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:48.081910 20272 raft_consensus.cc:3037] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:48.083034 20272 raft_consensus.cc:2454] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4e4906ca1ad49f9969a904b34950acf in term 1.
I20260811 07:38:48.083034 20118 raft_consensus.cc:2454] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4e4906ca1ad49f9969a904b34950acf in term 1.
I20260811 07:38:48.083308 20351 leader_election.cc:302] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c381f1ff26854195a6a1d5703cd086ca, e4e4906ca1ad49f9969a904b34950acf; no voters:
I20260811 07:38:48.083472 20513 raft_consensus.cc:2781] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:48.083676 20513 raft_consensus.cc:686] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 1 LEADER]: Becoming Leader. State: Replica: e4e4906ca1ad49f9969a904b34950acf, State: Running, Role: LEADER
I20260811 07:38:48.084008 20513 consensus_queue.cc:237] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.086299 19945 catalog_manager.cc:5495] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf reported cstate change: term changed from 0 to 1, leader changed from <none> to e4e4906ca1ad49f9969a904b34950acf (127.18.72.195). New cstate: current_term: 1 leader_uuid: "e4e4906ca1ad49f9969a904b34950acf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } health_report { overall_health: HEALTHY } } }
Aug 11 07:38:48 dist-test-slave-zkpd krb5kdc[19899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433926, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:48.146276 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.137773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44301 (local address 127.0.0.1:38322)
0811 07:38:48.138082 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:48.138094 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:48.138236 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:48.138510 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:48.138513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:48.138967 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:48.139066 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.139072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.139825 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.139837 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.140399 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.140406 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.140846 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.140850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.140929 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.141475 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:48.141487 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:48.142952 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:48.144634 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:48.144646 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:48.144653 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:48.144898 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:48.145174 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:48.145178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:48.145180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:48.145267 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:48.145640 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:48.145648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:48.145766 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 07:38:48.145887 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 07:38:48.146046 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":120,"threads_started":1}
I20260811 07:38:48.146276 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.137797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.18.72.254:44301)
0811 07:38:48.137922 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.137948 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.138175 (+ 227us) server_negotiation.cc:407] Connection header received
0811 07:38:48.138363 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.138366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.138403 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.138492 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:48.139194 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.139715 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.140530 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.140734 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.140839 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.143088 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:48.143094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:48.143100 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:48.143121 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:48.144524 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:48.144998 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:48.145003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:48.145005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:48.145083 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:48.145358 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:48.145361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:48.145363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:48.145534 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:48.145640 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.145947 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.146098 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":49}
I20260811 07:38:48.150516 19945 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:38322:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 07:38:48.151265 19945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:48.156103 20249 tablet_service.cc:1467] Processing CreateTablet for tablet c81c2c2bf9404d81b7f01468e265a8f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66e0fdca130047728855e286a7f777dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:38:48.156122 20095 tablet_service.cc:1467] Processing CreateTablet for tablet c81c2c2bf9404d81b7f01468e265a8f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66e0fdca130047728855e286a7f777dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:38:48.156122 20407 tablet_service.cc:1467] Processing CreateTablet for tablet c81c2c2bf9404d81b7f01468e265a8f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66e0fdca130047728855e286a7f777dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:38:48.156510 20249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81c2c2bf9404d81b7f01468e265a8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:48.156520 20407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81c2c2bf9404d81b7f01468e265a8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:48.156520 20095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81c2c2bf9404d81b7f01468e265a8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:48.159005 20504 tablet_bootstrap.cc:492] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: Bootstrap starting.
I20260811 07:38:48.159253 20505 tablet_bootstrap.cc:492] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: Bootstrap starting.
I20260811 07:38:48.159564 20506 tablet_bootstrap.cc:492] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: Bootstrap starting.
I20260811 07:38:48.160113 20505 tablet_bootstrap.cc:654] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:48.160125 20504 tablet_bootstrap.cc:654] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: Neither blocks nor log segments found. Creating new log.
W20260811 07:38:48.160308 20481 tablet.cc:2369] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:38:48.160530 20506 tablet_bootstrap.cc:654] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:48.161533 20504 tablet_bootstrap.cc:492] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: No bootstrap required, opened a new log
I20260811 07:38:48.161639 20504 ts_tablet_manager.cc:1397] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 07:38:48.161770 20506 tablet_bootstrap.cc:492] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: No bootstrap required, opened a new log
I20260811 07:38:48.161770 20505 tablet_bootstrap.cc:492] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: No bootstrap required, opened a new log
I20260811 07:38:48.161862 20506 ts_tablet_manager.cc:1397] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 07:38:48.161862 20505 ts_tablet_manager.cc:1397] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 07:38:48.162180 20504 raft_consensus.cc:348] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.162298 20504 raft_consensus.cc:374] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:48.162339 20504 raft_consensus.cc:729] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c381f1ff26854195a6a1d5703cd086ca, State: Initialized, Role: FOLLOWER
I20260811 07:38:48.162329 20505 raft_consensus.cc:348] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.162451 20505 raft_consensus.cc:374] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:48.162515 20505 raft_consensus.cc:729] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96bbdc97e0347bfb88bcfc65e00467c, State: Initialized, Role: FOLLOWER
I20260811 07:38:48.162499 20504 consensus_queue.cc:260] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.162612 20506 raft_consensus.cc:348] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.162649 20505 consensus_queue.cc:260] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.162813 20504 ts_tablet_manager.cc:1428] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 07:38:48.162751 20511 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.162909 20511 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:48.162935 20505 ts_tablet_manager.cc:1428] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:38:48.162874 20526 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.163000 20526 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:48.162730 20506 raft_consensus.cc:374] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:48.163775 20506 raft_consensus.cc:729] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e4906ca1ad49f9969a904b34950acf, State: Initialized, Role: FOLLOWER
I20260811 07:38:48.163954 20506 consensus_queue.cc:260] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.164218 20513 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.164312 20506 ts_tablet_manager.cc:1428] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 07:38:48.164355 20513 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:48.433233 20526 raft_consensus.cc:482] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:48.433427 20526 raft_consensus.cc:504] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.433971 20526 leader_election.cc:288] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681), e4e4906ca1ad49f9969a904b34950acf (127.18.72.195:42019)
I20260811 07:38:48.434473 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81c2c2bf9404d81b7f01468e265a8f6" candidate_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c381f1ff26854195a6a1d5703cd086ca" is_pre_election: true
I20260811 07:38:48.434548 20429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81c2c2bf9404d81b7f01468e265a8f6" candidate_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4e4906ca1ad49f9969a904b34950acf" is_pre_election: true
I20260811 07:38:48.434643 20272 raft_consensus.cc:2454] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a96bbdc97e0347bfb88bcfc65e00467c in term 0.
I20260811 07:38:48.434756 20429 raft_consensus.cc:2454] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a96bbdc97e0347bfb88bcfc65e00467c in term 0.
I20260811 07:38:48.434995 20044 leader_election.cc:302] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96bbdc97e0347bfb88bcfc65e00467c, c381f1ff26854195a6a1d5703cd086ca; no voters:
I20260811 07:38:48.435174 20526 raft_consensus.cc:2781] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:48.435236 20526 raft_consensus.cc:482] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:48.435276 20526 raft_consensus.cc:3037] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:48.436494 20526 raft_consensus.cc:504] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.436822 20526 leader_election.cc:288] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [CANDIDATE]: Term 1 election: Requested vote from peers c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681), e4e4906ca1ad49f9969a904b34950acf (127.18.72.195:42019)
I20260811 07:38:48.437166 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81c2c2bf9404d81b7f01468e265a8f6" candidate_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c381f1ff26854195a6a1d5703cd086ca"
I20260811 07:38:48.437222 20429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81c2c2bf9404d81b7f01468e265a8f6" candidate_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4e4906ca1ad49f9969a904b34950acf"
I20260811 07:38:48.437330 20272 raft_consensus.cc:3037] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:48.437330 20429 raft_consensus.cc:3037] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:48.438465 20429 raft_consensus.cc:2454] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a96bbdc97e0347bfb88bcfc65e00467c in term 1.
I20260811 07:38:48.438465 20272 raft_consensus.cc:2454] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a96bbdc97e0347bfb88bcfc65e00467c in term 1.
I20260811 07:38:48.438781 20044 leader_election.cc:302] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96bbdc97e0347bfb88bcfc65e00467c, c381f1ff26854195a6a1d5703cd086ca; no voters:
I20260811 07:38:48.438954 20526 raft_consensus.cc:2781] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:48.439133 20526 raft_consensus.cc:686] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [term 1 LEADER]: Becoming Leader. State: Replica: a96bbdc97e0347bfb88bcfc65e00467c, State: Running, Role: LEADER
I20260811 07:38:48.439304 20526 consensus_queue.cc:237] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } }
I20260811 07:38:48.439961 20549 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: TxnStatusTablet state changed. Reason: New leader a96bbdc97e0347bfb88bcfc65e00467c. Latest consensus state: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.440086 20549 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:38:48.440313 20551 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 07:38:48.440516 20551 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 07:38:48.441217 19946 catalog_manager.cc:5495] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c reported cstate change: term changed from 0 to 1, leader changed from <none> to a96bbdc97e0347bfb88bcfc65e00467c (127.18.72.193). New cstate: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } health_report { overall_health: UNKNOWN } } }
I20260811 07:38:48.462733 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.459103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.18.72.193:39441)
0811 07:38:48.459240 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.459247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.459306 (+ 59us) server_negotiation.cc:407] Connection header received
0811 07:38:48.459418 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.459420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.459464 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.459527 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:48.459819 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.460467 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.461649 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.461855 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.461922 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.462227 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:48.462314 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:48.462373 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.462505 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.462551 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:38:48.462725 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.459014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39441 (local address 127.0.0.1:52312)
0811 07:38:48.459169 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:48.459184 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:48.459310 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:48.459572 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:48.459575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:48.459583 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:38:48.459699 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.459705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.460583 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.460587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.461530 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.461541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.461952 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.461956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.462044 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.462099 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:38:48.462401 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:38:48.462416 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 07:38:48.462484 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 07:38:48.462589 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 07:38:48.465551 20429 raft_consensus.cc:1261] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Refusing update from remote peer a96bbdc97e0347bfb88bcfc65e00467c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:48.465636 20272 raft_consensus.cc:1261] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Refusing update from remote peer a96bbdc97e0347bfb88bcfc65e00467c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:48.466079 20549 consensus_queue.cc:1035] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:48.466230 20526 consensus_queue.cc:1035] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:48.467639 20513 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: TxnStatusTablet state changed. Reason: New leader a96bbdc97e0347bfb88bcfc65e00467c. Latest consensus state: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.467749 20513 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:48.467940 20511 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: TxnStatusTablet state changed. Reason: New leader a96bbdc97e0347bfb88bcfc65e00467c. Latest consensus state: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.468052 20511 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:48.469038 20549 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.469138 20549 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:38:48.469581 20526 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.469669 20526 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P a96bbdc97e0347bfb88bcfc65e00467c: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:38:48.470482 20511 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.470592 20511 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P c381f1ff26854195a6a1d5703cd086ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:48.470728 20513 tablet_replica.cc:439] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 } } peers { permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 } } peers { permanent_uuid: "e4e4906ca1ad49f9969a904b34950acf" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42019 } } }
I20260811 07:38:48.470834 20513 tablet_replica.cc:442] T c81c2c2bf9404d81b7f01468e265a8f6 P e4e4906ca1ad49f9969a904b34950acf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:48.481573 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.477651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41228 (local address 127.18.72.195:42019)
0811 07:38:48.477861 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.477865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.477877 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:38:48.478040 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.478042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.478093 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.478194 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:48.478451 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.478956 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.480147 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.480368 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.480446 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.480727 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:48.481158 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:48.481245 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.481344 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.481389 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260811 07:38:48.491823 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.488478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:43279 (local address 127.18.72.193:39441)
0811 07:38:48.488644 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.488647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.488723 (+ 76us) server_negotiation.cc:407] Connection header received
0811 07:38:48.488881 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.488883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.488927 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.488987 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:48.489255 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.489730 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.490874 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.491067 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.491137 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.491437 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:48.491499 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:48.491545 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.491660 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.491722 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 07:38:48.491921 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.488333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39441 (local address 127.18.72.195:43279)
0811 07:38:48.488648 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:48.488664 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:48.488772 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:48.489022 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:48.489025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:48.489034 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:38:48.489148 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.489153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.489830 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.489833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.490754 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.490765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.491165 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.491169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.491258 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.491309 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:38:48.491573 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:38:48.491579 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:38:48.491644 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 07:38:48.491737 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":119,"threads_started":1}
I20260811 07:38:48.498800 20272 raft_consensus.cc:1261] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Refusing update from remote peer e4e4906ca1ad49f9969a904b34950acf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:48.499126 20118 raft_consensus.cc:1261] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Refusing update from remote peer e4e4906ca1ad49f9969a904b34950acf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:48.499269 20513 consensus_queue.cc:1035] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "c381f1ff26854195a6a1d5703cd086ca" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 33681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:48.499639 20513 consensus_queue.cc:1035] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96bbdc97e0347bfb88bcfc65e00467c" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:48.523232 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.518832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:59743 (local address 127.18.72.195:42019)
0811 07:38:48.518965 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.518970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.519500 (+ 530us) server_negotiation.cc:407] Connection header received
0811 07:38:48.519633 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.519636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.519687 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.519762 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:48.520126 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.520858 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.521987 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.522303 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.522429 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.522762 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:48.522874 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:48.522944 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.523046 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.523099 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:38:48.523284 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.518794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42019 (local address 127.18.72.193:59743)
0811 07:38:48.519401 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:48.519416 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:48.519527 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:48.519792 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:48.519794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:48.519804 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:38:48.519967 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.519978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.520982 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.520988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.521869 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:48.521880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.522427 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.522431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:48.522533 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.522589 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:38:48.522966 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:38:48.522973 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:38:48.523037 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 07:38:48.523132 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":114,"threads_started":1}
I20260811 07:38:48.547142 19944 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:48.547334 19944 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:48.549743 19944 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98c317794dd84712a50b2d18b38711e0: Sending DeleteTablet for 3 replicas of tablet 359c499e71a64120b8c24527127c86f7
I20260811 07:38:48.550484 20249 tablet_service.cc:1514] Processing DeleteTablet for tablet 359c499e71a64120b8c24527127c86f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:48 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:36966
I20260811 07:38:48.550501 20407 tablet_service.cc:1514] Processing DeleteTablet for tablet 359c499e71a64120b8c24527127c86f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:48 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:41216
I20260811 07:38:48.550958 20578 tablet_replica.cc:330] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: stopping tablet replica
I20260811 07:38:48.551117 20578 raft_consensus.cc:2229] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:38:48.551121 20095 tablet_service.cc:1514] Processing DeleteTablet for tablet 359c499e71a64120b8c24527127c86f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:48 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:52302
I20260811 07:38:48.551501 20578 raft_consensus.cc:2258] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:48.552264 20578 ts_tablet_manager.cc:1905] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:48.552619 20580 tablet_replica.cc:330] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: stopping tablet replica
I20260811 07:38:48.552757 20580 raft_consensus.cc:2229] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:48.552891 20580 raft_consensus.cc:2258] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:48.553136 20579 tablet_replica.cc:330] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: stopping tablet replica
I20260811 07:38:48.553261 20579 raft_consensus.cc:2229] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:48.553380 20579 raft_consensus.cc:2258] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:48.553937 20580 ts_tablet_manager.cc:1905] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:48.554032 20579 ts_tablet_manager.cc:1905] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:48.556932 20579 ts_tablet_manager.cc:1918] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:48.557019 20579 log.cc:1196] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/359c499e71a64120b8c24527127c86f7
I20260811 07:38:48.557301 20579 ts_tablet_manager.cc:1939] T 359c499e71a64120b8c24527127c86f7 P c381f1ff26854195a6a1d5703cd086ca: Deleting consensus metadata
I20260811 07:38:48.558089 19930 catalog_manager.cc:4841] TS c381f1ff26854195a6a1d5703cd086ca (127.18.72.194:33681): tablet 359c499e71a64120b8c24527127c86f7 (table test-table [id=f8636902f42b4864b1ac1d25e736e2c3]) successfully deleted
I20260811 07:38:48.559440 20578 ts_tablet_manager.cc:1918] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:48.559525 20578 log.cc:1196] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/359c499e71a64120b8c24527127c86f7
I20260811 07:38:48.559780 20578 ts_tablet_manager.cc:1939] T 359c499e71a64120b8c24527127c86f7 P e4e4906ca1ad49f9969a904b34950acf: Deleting consensus metadata
I20260811 07:38:48.560488 19930 catalog_manager.cc:4841] TS e4e4906ca1ad49f9969a904b34950acf (127.18.72.195:42019): tablet 359c499e71a64120b8c24527127c86f7 (table test-table [id=f8636902f42b4864b1ac1d25e736e2c3]) successfully deleted
I20260811 07:38:48.560781 20580 ts_tablet_manager.cc:1918] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:48.560865 20580 log.cc:1196] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/359c499e71a64120b8c24527127c86f7
I20260811 07:38:48.561129 20580 ts_tablet_manager.cc:1939] T 359c499e71a64120b8c24527127c86f7 P a96bbdc97e0347bfb88bcfc65e00467c: Deleting consensus metadata
I20260811 07:38:48.561781 19933 catalog_manager.cc:4841] TS a96bbdc97e0347bfb88bcfc65e00467c (127.18.72.193:39441): tablet 359c499e71a64120b8c24527127c86f7 (table test-table [id=f8636902f42b4864b1ac1d25e736e2c3]) successfully deleted
Aug 11 07:38:48 dist-test-slave-zkpd krb5kdc[19899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433927, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:38:48.572501 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.557573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.18.72.193:39441)
0811 07:38:48.557683 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.557688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.560165 (+ 2477us) server_negotiation.cc:407] Connection header received
0811 07:38:48.560350 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.560353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.560401 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.560450 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:48.561636 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.562434 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.563735 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.564135 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.564209 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.568265 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:48.568277 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:48.568296 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:48.568324 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:48.570441 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:48.570982 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:48.570989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:48.570990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:48.571084 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:48.571411 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:48.571414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:48.571416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:48.571714 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:48.571965 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.572199 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.572321 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 07:38:48.573292 20162 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52318
I20260811 07:38:48.584964 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:48.575148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.18.72.254:44301)
0811 07:38:48.575265 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:48.575269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:48.576271 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 07:38:48.576367 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:48.576370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:48.576423 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:48.576508 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:48.577296 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.578101 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.579254 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:48.579599 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:48.579703 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:48.581348 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:48.581352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:48.581355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:48.581384 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:48.583162 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:48.583643 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:48.583646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:48.583648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:48.583733 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:48.584039 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:48.584042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:48.584044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:48.584265 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:48.584392 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:48.584651 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 07:38:48.584768 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 07:38:48.585577 19944 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38332
I20260811 07:38:48.587116 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 20027
I20260811 07:38:48.605436 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 20180
I20260811 07:38:48.614524 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 20332
I20260811 07:38:48.622004 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 19913
2026-08-11T07:38:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3073 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:38:48 dist-test-slave-zkpd krb5kdc[20587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:38:48 dist-test-slave-zkpd krb5kdc[20587](info): set up 2 sockets
Aug 11 07:38:48 dist-test-slave-zkpd krb5kdc[20587](info): commencing operation
krb5kdc: starting...
W20260811 07:38:49.708956 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:38:49 dist-test-slave-zkpd krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:38:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:38:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:38:49.832336 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:33815
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:33815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:49.944783 20601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:49.945029 20601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:49.945070 20601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:49.948951 20601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:38:49.949002 20601 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 07:38:49.949019 20601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:49.949038 20601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:49.949054 20601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:38:49.949069 20601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:38:49.954170 20601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:33815
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:33815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:49.955099 20601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:49.956144 20601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:49.964761 20606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:49.964756 20609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:49.964757 20607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:49.965095 20601 server_base.cc:1034] running on GCE node
I20260811 07:38:49.965667 20601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:49.966794 20601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:49.967978 20601 hybrid_clock.cc:648] HybridClock initialized: now 1786433929967949 us; error 36 us; skew 500 ppm
Aug 11 07:38:49 dist-test-slave-zkpd krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433929, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:49.970677 20601 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:38:49.971537 20601 webserver.cc:460] Webserver started at http://127.18.72.254:35041/ using document root <none> and password file <none>
I20260811 07:38:49.972193 20601 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:49.972285 20601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:49.972519 20601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:49.974563 20601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "48c36784ecdb4b8897db0e621f97c1b6"
format_stamp: "Formatted at 2026-08-11 07:38:49 on dist-test-slave-zkpd"
server_key: "94e12eb85c1de0bc41312460862fc6cc"
server_key_iv: "0c95f3ec006b588edab71c5662b21997"
server_key_version: "encryptionkey@0"
I20260811 07:38:49.975142 20601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "48c36784ecdb4b8897db0e621f97c1b6"
format_stamp: "Formatted at 2026-08-11 07:38:49 on dist-test-slave-zkpd"
server_key: "94e12eb85c1de0bc41312460862fc6cc"
server_key_iv: "0c95f3ec006b588edab71c5662b21997"
server_key_version: "encryptionkey@0"
I20260811 07:38:49.978854 20601 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:38:49.981242 20617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:49.982430 20601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:38:49.982599 20601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "48c36784ecdb4b8897db0e621f97c1b6"
format_stamp: "Formatted at 2026-08-11 07:38:49 on dist-test-slave-zkpd"
server_key: "94e12eb85c1de0bc41312460862fc6cc"
server_key_iv: "0c95f3ec006b588edab71c5662b21997"
server_key_version: "encryptionkey@0"
I20260811 07:38:49.982712 20601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:50.018363 20601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:50.021587 20601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:50.021798 20601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:50.030216 20601 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:33815
I20260811 07:38:50.030236 20691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:33815 every 8 connection(s)
I20260811 07:38:50.031193 20601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:38:50.034111 20692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:50.038761 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 20601
I20260811 07:38:50.038877 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:38:50.039161 18723 external_mini_cluster.cc:1428] Setting key becb04927637ca966b1b0e4aac05ece6
I20260811 07:38:50.040081 20692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: Bootstrap starting.
I20260811 07:38:50.042505 20692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:50.043286 20692 log.cc:824] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:50.045281 20692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: No bootstrap required, opened a new log
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:50.049552 20692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c36784ecdb4b8897db0e621f97c1b6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33815 } }
I20260811 07:38:50.049849 20692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:50.049952 20692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c36784ecdb4b8897db0e621f97c1b6, State: Initialized, Role: FOLLOWER
I20260811 07:38:50.050434 20692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c36784ecdb4b8897db0e621f97c1b6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33815 } }
I20260811 07:38:50.050561 20692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:38:50.050642 20692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:38:50.050737 20692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:50.051873 20692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c36784ecdb4b8897db0e621f97c1b6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33815 } }
I20260811 07:38:50.052325 20692 leader_election.cc:302] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48c36784ecdb4b8897db0e621f97c1b6; no voters:
I20260811 07:38:50.052702 20692 leader_election.cc:288] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:38:50.052770 20697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:50.053050 20697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [term 1 LEADER]: Becoming Leader. State: Replica: 48c36784ecdb4b8897db0e621f97c1b6, State: Running, Role: LEADER
I20260811 07:38:50.053417 20697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c36784ecdb4b8897db0e621f97c1b6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33815 } }
I20260811 07:38:50.054056 20692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:38:50.054286 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.040606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48716 (local address 127.18.72.254:33815)
0811 07:38:50.041043 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.041059 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.041095 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:38:50.041736 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.041752 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.042091 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.042455 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.043285 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.044436 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.045291 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.045823 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.045994 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.048813 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.048818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.048832 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.048859 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.050707 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.051213 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.051218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.051223 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.051340 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.051638 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.051641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.051642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.052161 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.052309 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.052979 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.053277 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":142,"threads_started":1}
I20260811 07:38:50.055255 20698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48c36784ecdb4b8897db0e621f97c1b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c36784ecdb4b8897db0e621f97c1b6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33815 } } }
I20260811 07:38:50.055409 20698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:50.055255 20699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48c36784ecdb4b8897db0e621f97c1b6. Latest consensus state: current_term: 1 leader_uuid: "48c36784ecdb4b8897db0e621f97c1b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c36784ecdb4b8897db0e621f97c1b6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33815 } } }
I20260811 07:38:50.055473 20699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:50.056597 20706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:38:50.059460 20706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:38:50.064683 20706 catalog_manager.cc:1347] Generated new cluster ID: 0242d0e490074de4a0dc21510a427986
I20260811 07:38:50.064749 20706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:38:50.084640 20706 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:38:50.085707 20706 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:38:50.091799 20706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: Generated new TSK 0
I20260811 07:38:50.092572 20706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:38:50.124341 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33815
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:50.238854 20719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:50.239095 20719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:50.239171 20719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:50.243165 20719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:50.243224 20719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:50.243264 20719 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 07:38:50.243283 20719 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:38:50.243359 20719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:38:50.248592 20719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:50.249650 20719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:50.250706 20719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:50.260281 20724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:50.260303 20725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:50.260641 20719 server_base.cc:1034] running on GCE node
W20260811 07:38:50.260315 20727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:50.261144 20719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:50.261790 20719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:50.262984 20719 hybrid_clock.cc:648] HybridClock initialized: now 1786433930262969 us; error 27 us; skew 500 ppm
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433930, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:50.265738 20719 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:38:50.266562 20719 webserver.cc:460] Webserver started at http://127.18.72.193:37035/ using document root <none> and password file <none>
I20260811 07:38:50.267134 20719 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:50.267223 20719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:50.267454 20719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:50.269500 20719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "075ac11054434b95aff06b492d2266aa"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "7331d2e9d1cb9f9cc20c7f4c80cd4ec2"
server_key_iv: "f6300b56c23db6528890ccd4444d7a14"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.270071 20719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "075ac11054434b95aff06b492d2266aa"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "7331d2e9d1cb9f9cc20c7f4c80cd4ec2"
server_key_iv: "f6300b56c23db6528890ccd4444d7a14"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.273806 20719 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:38:50.276239 20734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:50.277472 20719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:38:50.277608 20719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "075ac11054434b95aff06b492d2266aa"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "7331d2e9d1cb9f9cc20c7f4c80cd4ec2"
server_key_iv: "f6300b56c23db6528890ccd4444d7a14"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.277721 20719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:50.295250 20719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:50.298288 20719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:50.298509 20719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:50.300192 20719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:50.300266 20719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:50.300361 20719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:50.300417 20719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433930, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:50.312254 20719 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:40643
I20260811 07:38:50.312316 20860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:40643 every 8 connection(s)
I20260811 07:38:50.313439 20719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:38:50.314788 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.301053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:36607 (local address 127.18.72.254:33815)
0811 07:38:50.301178 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.301182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.302289 (+ 1107us) server_negotiation.cc:407] Connection header received
0811 07:38:50.303300 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.303304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.303375 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.303474 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.304891 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.305853 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.307096 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.307463 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.307590 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.310537 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.310541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.310543 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.310572 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.312293 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.312813 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.312817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.312818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.312917 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.313271 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.313275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.313276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.313536 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.313637 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.314560 (+ 923us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.314670 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:38:50.315387 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.301624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33815 (local address 127.18.72.193:36607)
0811 07:38:50.302122 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.302163 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.303034 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.303854 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.303863 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.304263 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:50.304725 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.304737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.305994 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.305998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.306963 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.306974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.307643 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.307647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.307835 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.308489 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:50.308515 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:50.310363 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:50.312418 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.312425 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.312440 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.312706 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.313016 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.313018 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.313019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.313145 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.313707 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:50.313714 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:50.314054 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.314304 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.314574 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":140,"threads_started":1}
I20260811 07:38:50.316704 20861 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33815
I20260811 07:38:50.316876 20861 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:50.317358 20861 heartbeater.cc:511] Master 127.18.72.254:33815 requested a full tablet report, sending...
I20260811 07:38:50.318943 20634 ts_manager.cc:194] Registered new tserver with Master: 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643)
I20260811 07:38:50.320279 20634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:36607
I20260811 07:38:50.321188 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 20719
I20260811 07:38:50.321336 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:38:50.321662 18723 external_mini_cluster.cc:1428] Setting key 591bf8c3fbe1b5b6e8265566aae764e8
I20260811 07:38:50.327560 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.318836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33815 (local address 127.18.72.193:48541)
0811 07:38:50.319141 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.319156 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.319242 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.320253 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.320256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.320473 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:50.320593 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.320601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.321630 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.321633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.322413 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.322424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.322973 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.322977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.323078 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.323648 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:50.323660 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:50.324093 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:50.325932 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.325938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.325941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.326260 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.326569 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.326575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.326578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.326692 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.327127 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:50.327129 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:50.327209 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.327295 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.327411 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":41,"thread_start_us":125,"threads_started":1}
I20260811 07:38:50.327746 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.318979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:48541 (local address 127.18.72.254:33815)
0811 07:38:50.320060 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.320063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.320074 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:38:50.320105 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.320107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.320141 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.320222 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.320753 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.321497 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.322538 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.322853 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.322968 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.324231 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.324238 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.324244 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.324276 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.325812 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.326382 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.326386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.326387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.326473 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.326786 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.326789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.326790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.327028 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.327132 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.327380 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.327516 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002}
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:38:50.349053 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33815
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:50.464519 20870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:50.464807 20870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:50.464886 20870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:50.468870 20870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:50.468950 20870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:50.469022 20870 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 07:38:50.469064 20870 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:38:50.469127 20870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:38:50.474396 20870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:50.475414 20870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:50.476557 20870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:50.486394 20875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:50.486434 20878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:50.486490 20870 server_base.cc:1034] running on GCE node
W20260811 07:38:50.486394 20876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:50.487058 20870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:50.487689 20870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:50.488852 20870 hybrid_clock.cc:648] HybridClock initialized: now 1786433930488850 us; error 51 us; skew 500 ppm
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433930, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:50.491436 20870 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:38:50.492395 20870 webserver.cc:460] Webserver started at http://127.18.72.194:45941/ using document root <none> and password file <none>
I20260811 07:38:50.492967 20870 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:50.493060 20870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:50.493300 20870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:50.495357 20870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "1299e624f3f04d25988b35c8d78e0e4c"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "52abc8f79d391363774d114ec932a6d6"
server_key_iv: "84570fa5663e508039a9aadbd40e54f9"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.495965 20870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "1299e624f3f04d25988b35c8d78e0e4c"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "52abc8f79d391363774d114ec932a6d6"
server_key_iv: "84570fa5663e508039a9aadbd40e54f9"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.499653 20870 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:38:50.502049 20886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:50.503243 20870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:38:50.503387 20870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "1299e624f3f04d25988b35c8d78e0e4c"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "52abc8f79d391363774d114ec932a6d6"
server_key_iv: "84570fa5663e508039a9aadbd40e54f9"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.503497 20870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:50.519287 20870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:50.522017 20870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:50.522217 20870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:50.523669 20870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:50.523739 20870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:50.523808 20870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:50.523845 20870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433930, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:50.535180 20870 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:40353
I20260811 07:38:50.535269 21017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:40353 every 8 connection(s)
I20260811 07:38:50.536319 20870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:38:50.537804 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.524394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:43365 (local address 127.18.72.254:33815)
0811 07:38:50.524516 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.524523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.526119 (+ 1596us) server_negotiation.cc:407] Connection header received
0811 07:38:50.527294 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.527297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.527366 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.527459 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.529154 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.529770 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.531077 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.531344 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.531464 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.533992 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.533996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.533999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.534028 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.535667 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.536204 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.536210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.536215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.536314 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.536687 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.536693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.536697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.536963 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.537072 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.537583 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.537687 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:38:50.538542 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.525446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33815 (local address 127.18.72.194:43365)
0811 07:38:50.525976 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.526017 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.527080 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.527676 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.527686 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.528231 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:50.528964 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.528987 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.529914 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.529917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.530949 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.530966 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.531516 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.531521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.531692 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.532269 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:50.532285 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:50.533785 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:50.535777 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.535781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.535792 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.536104 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.536422 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.536426 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.536428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.536570 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.537123 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:50.537129 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:50.537351 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.537499 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.537734 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":125,"threads_started":1}
I20260811 07:38:50.539763 21018 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33815
I20260811 07:38:50.539932 21018 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:50.540511 21018 heartbeater.cc:511] Master 127.18.72.254:33815 requested a full tablet report, sending...
I20260811 07:38:50.541639 20636 ts_manager.cc:194] Registered new tserver with Master: 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194:40353)
I20260811 07:38:50.542361 20636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:43365
I20260811 07:38:50.545596 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 20870
I20260811 07:38:50.545732 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:38:50.546077 18723 external_mini_cluster.cc:1428] Setting key 7881e2ddb71339495d673b64e3188cfc
I20260811 07:38:50.549409 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.542244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:45861 (local address 127.18.72.254:33815)
0811 07:38:50.542373 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.542376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.542499 (+ 123us) server_negotiation.cc:407] Connection header received
0811 07:38:50.542677 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.542681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.542729 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.542807 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.543229 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.543733 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.544682 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.544917 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.545027 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.546179 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.546187 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.546189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.546211 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.547683 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.548175 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.548179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.548181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.548263 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.548547 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.548552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.548554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.548771 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.548888 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.549150 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.549259 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:38:50.549433 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.542109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33815 (local address 127.18.72.194:45861)
0811 07:38:50.542413 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.542429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.542566 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.542826 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.542829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.543024 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:50.543117 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.543122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.543860 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.543864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.544559 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.544569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.545029 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.545033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.545122 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.545585 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:50.545600 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:50.546023 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:50.547813 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.547817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.547819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.548075 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.548361 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.548364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.548366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.548455 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.548880 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:50.548883 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:50.548986 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.549094 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.549250 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:38:50.576894 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33815
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:50.692744 21029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:50.692998 21029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:50.693084 21029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:50.697048 21029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:50.697099 21029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:50.697163 21029 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 07:38:50.697198 21029 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:38:50.697253 21029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:38:50.702374 21029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:50.703471 21029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:50.704538 21029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:50.714035 21038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:50.714035 21036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:50.714048 21040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:50.714337 21029 server_base.cc:1034] running on GCE node
I20260811 07:38:50.714762 21029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:50.715382 21029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:50.716560 21029 hybrid_clock.cc:648] HybridClock initialized: now 1786433930716525 us; error 47 us; skew 500 ppm
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433930, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:50.719247 21029 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:38:50.720166 21029 webserver.cc:460] Webserver started at http://127.18.72.195:44681/ using document root <none> and password file <none>
I20260811 07:38:50.720734 21029 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:50.720824 21029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:50.721063 21029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:50.723057 21029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "1dfa28127e504428b496ba7f62c43d98"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "597f255f86abda0e794eb08bb1801bfc"
server_key_iv: "51ed32f7936fcd0a486cce72b16d6698"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.723636 21029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "1dfa28127e504428b496ba7f62c43d98"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "597f255f86abda0e794eb08bb1801bfc"
server_key_iv: "51ed32f7936fcd0a486cce72b16d6698"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.727495 21029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 07:38:50.729897 21049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:50.731096 21029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 07:38:50.731230 21029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "1dfa28127e504428b496ba7f62c43d98"
format_stamp: "Formatted at 2026-08-11 07:38:50 on dist-test-slave-zkpd"
server_key: "597f255f86abda0e794eb08bb1801bfc"
server_key_iv: "51ed32f7936fcd0a486cce72b16d6698"
server_key_version: "encryptionkey@0"
I20260811 07:38:50.731374 21029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:50.756564 21029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:50.759438 21029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:50.759665 21029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:50.761612 21029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:50.761695 21029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:50.761787 21029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:50.761814 21029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433930, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:50.773082 21029 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:41133
I20260811 07:38:50.773138 21181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:41133 every 8 connection(s)
I20260811 07:38:50.774019 21029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:38:50.776278 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.762293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:51849 (local address 127.18.72.254:33815)
0811 07:38:50.762414 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.762418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.764152 (+ 1734us) server_negotiation.cc:407] Connection header received
0811 07:38:50.765314 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.765317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.765390 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.765494 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.766711 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.767310 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.769211 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.769518 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.769638 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.772311 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.772315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.772319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.772357 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.774178 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.774803 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.774807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.774809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.774917 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.775261 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.775263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.775264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.775447 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.775526 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.776052 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.776160 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 07:38:50.777439 21059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.763428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33815 (local address 127.18.72.195:51849)
0811 07:38:50.764005 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.764048 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.765091 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.765674 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.765682 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.766090 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:50.766559 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.766573 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.767454 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.767461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.769066 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.769081 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.769674 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.769679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.769860 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.770443 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:50.770460 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:50.772132 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:50.774299 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.774305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.774321 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.774700 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.775024 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.775028 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.775030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.775181 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.775536 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:50.775540 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:50.775767 (+ 227us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.776007 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.776270 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":139,"threads_started":1}
I20260811 07:38:50.778767 21182 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33815
I20260811 07:38:50.778927 21182 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:50.779484 21182 heartbeater.cc:511] Master 127.18.72.254:33815 requested a full tablet report, sending...
I20260811 07:38:50.780675 20636 ts_manager.cc:194] Registered new tserver with Master: 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
I20260811 07:38:50.781466 20636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:51849
I20260811 07:38:50.784184 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 21029
I20260811 07:38:50.784307 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:38:50.784653 18723 external_mini_cluster.cc:1428] Setting key 73550f75ac81f02453649aa19baa31d6
I20260811 07:38:50.787060 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:38:50.788823 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.781138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33815 (local address 127.18.72.195:54843)
0811 07:38:50.781492 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.781504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.781661 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.781962 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.781965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.782194 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:50.782306 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.782313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.783064 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.783068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.783801 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.783811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.784426 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.784430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.784521 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.785036 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:50.785051 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:50.785422 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:50.787352 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.787355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.787357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.787542 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.787841 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.787843 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.787844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.787966 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.788429 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:50.788431 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:50.788505 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.788586 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.788692 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":85,"threads_started":1}
I20260811 07:38:50.789045 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.781228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:54843 (local address 127.18.72.254:33815)
0811 07:38:50.781361 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.781364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.781587 (+ 223us) server_negotiation.cc:407] Connection header received
0811 07:38:50.781793 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.781796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.781850 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.781937 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.782449 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.782936 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.783952 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.784280 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.784401 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.785522 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.785526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.785529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.785556 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.787242 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.787639 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.787643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.787645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.787744 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.788073 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.788079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.788083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.788330 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.788441 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.788674 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.788806 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:50.814354 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.806208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.18.72.254:33815)
0811 07:38:50.806354 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.806358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.806371 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:38:50.806411 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.806413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.806451 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.806542 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.807259 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.807785 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.808650 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.808873 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.808986 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.811348 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.811351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.811353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.811374 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.812663 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.813104 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.813112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.813113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.813187 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.813474 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.813477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.813479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.813660 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.813810 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.814064 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.814176 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:38:50.817242 20636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:38:50.820429 20636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:50.835422 21206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.830364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.18.72.195:41133)
0811 07:38:50.830996 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.831015 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.831033 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:38:50.831097 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.831101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.831246 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.831432 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:50.831627 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.832593 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.834021 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.834603 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.834740 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.834917 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.835140 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.835253 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":122,"threads_started":1}
I20260811 07:38:50.835511 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.829126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:40643 (local address 127.0.0.1:45636)
0811 07:38:50.829515 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.829567 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.829697 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.830503 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.830510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.830539 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:50.830716 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.830724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.832271 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.832274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.833752 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.833763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.834891 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.834895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.834990 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.835023 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.835079 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.835337 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":142,"thread_start_us":101,"threads_started":1}
I20260811 07:38:50.835575 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.829252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:41133 (local address 127.0.0.1:37574)
0811 07:38:50.830638 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.830655 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.830747 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.831400 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.831404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.831421 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:50.831525 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.831535 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.832728 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.832732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.833903 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.833911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.834784 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.834788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.834918 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.834968 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.835106 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.835429 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1325,"mutex_wait_us":50,"thread_start_us":94,"threads_started":1}
I20260811 07:38:50.835688 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.829241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.18.72.193:40643)
0811 07:38:50.829980 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.829988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.830013 (+ 25us) server_negotiation.cc:407] Connection header received
0811 07:38:50.830095 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.830100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.830319 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.830907 (+ 588us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:50.831055 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.832152 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.833897 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.834700 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.835266 (+ 566us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.835409 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.835460 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.835559 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":89,"threads_started":1}
I20260811 07:38:50.836005 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.829915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:40353 (local address 127.0.0.1:43400)
0811 07:38:50.830751 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.830766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.830853 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.831513 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.831516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.831533 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:50.831611 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.831617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.832844 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.832848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.833776 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.833787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.835094 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.835097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.835183 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.835215 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.835282 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.835361 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260811 07:38:50.836606 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.830103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.18.72.194:40353)
0811 07:38:50.830515 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.830526 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.830947 (+ 421us) server_negotiation.cc:407] Connection header received
0811 07:38:50.831031 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.831037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.831229 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.831381 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:50.831735 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.832725 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.833942 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.834756 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.834863 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.835034 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.836382 (+ 1348us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.836474 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":104,"threads_started":1}
I20260811 07:38:50.838516 21105 tablet_service.cc:1467] Processing CreateTablet for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c (DEFAULT_TABLE table=test-table [id=a853e25d47b24a3a8d9cef1c9bbbf7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:50.839150 20939 tablet_service.cc:1467] Processing CreateTablet for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c (DEFAULT_TABLE table=test-table [id=a853e25d47b24a3a8d9cef1c9bbbf7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:50.839600 21105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:50.839140 20792 tablet_service.cc:1467] Processing CreateTablet for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c (DEFAULT_TABLE table=test-table [id=a853e25d47b24a3a8d9cef1c9bbbf7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:50.840229 20792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:50.840257 20939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:50.846326 21207 tablet_bootstrap.cc:492] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Bootstrap starting.
I20260811 07:38:50.847675 21208 tablet_bootstrap.cc:492] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Bootstrap starting.
I20260811 07:38:50.848799 21207 tablet_bootstrap.cc:654] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:50.849673 21207 log.cc:824] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:50.849804 21208 tablet_bootstrap.cc:654] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:50.850436 21209 tablet_bootstrap.cc:492] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Bootstrap starting.
I20260811 07:38:50.850623 21208 log.cc:824] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:50.851842 21207 tablet_bootstrap.cc:492] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: No bootstrap required, opened a new log
I20260811 07:38:50.852078 21207 ts_tablet_manager.cc:1397] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:38:50.852437 21209 tablet_bootstrap.cc:654] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:50.852571 21208 tablet_bootstrap.cc:492] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: No bootstrap required, opened a new log
I20260811 07:38:50.852771 21208 ts_tablet_manager.cc:1397] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 07:38:50.853236 21209 log.cc:824] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:50.855347 21209 tablet_bootstrap.cc:492] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: No bootstrap required, opened a new log
I20260811 07:38:50.855535 21209 ts_tablet_manager.cc:1397] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:38:50.855868 21207 raft_consensus.cc:348] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.856163 21207 raft_consensus.cc:374] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:50.856272 21207 raft_consensus.cc:729] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfa28127e504428b496ba7f62c43d98, State: Initialized, Role: FOLLOWER
I20260811 07:38:50.856765 21207 consensus_queue.cc:260] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.856832 21208 raft_consensus.cc:348] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.857101 21208 raft_consensus.cc:374] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:50.857172 21208 raft_consensus.cc:729] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075ac11054434b95aff06b492d2266aa, State: Initialized, Role: FOLLOWER
I20260811 07:38:50.857570 21182 heartbeater.cc:503] Master 127.18.72.254:33815 was elected leader, sending a full tablet report...
I20260811 07:38:50.857621 21208 consensus_queue.cc:260] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.858017 21207 ts_tablet_manager.cc:1428] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260811 07:38:50.858486 21208 ts_tablet_manager.cc:1428] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 07:38:50.858804 20861 heartbeater.cc:503] Master 127.18.72.254:33815 was elected leader, sending a full tablet report...
I20260811 07:38:50.859714 21209 raft_consensus.cc:348] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.860018 21209 raft_consensus.cc:374] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:50.860086 21209 raft_consensus.cc:729] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Initialized, Role: FOLLOWER
I20260811 07:38:50.860509 21209 consensus_queue.cc:260] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.861191 21018 heartbeater.cc:503] Master 127.18.72.254:33815 was elected leader, sending a full tablet report...
I20260811 07:38:50.861428 21209 ts_tablet_manager.cc:1428] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:38:50.872879 21213 raft_consensus.cc:482] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:50.873075 21213 raft_consensus.cc:504] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.874125 21213 leader_election.cc:288] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194:40353)
I20260811 07:38:50.878604 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.874484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:54597 (local address 127.18.72.194:40353)
0811 07:38:50.874611 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.874615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.874764 (+ 149us) server_negotiation.cc:407] Connection header received
0811 07:38:50.874888 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.874891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.874933 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.875008 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:50.875289 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.875780 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.877485 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.877945 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.878036 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.878103 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.878379 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.878460 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 07:38:50.878790 21059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.874311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:40643 (local address 127.18.72.195:36431)
0811 07:38:50.874480 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.874496 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.874599 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.874886 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.874889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.874909 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:50.875058 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.875066 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.875872 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.875877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.877502 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.877513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.878346 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.878353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.878453 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.878474 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.878542 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.878606 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":37}
I20260811 07:38:50.879037 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.874333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:40353 (local address 127.18.72.195:54597)
0811 07:38:50.874673 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.874687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.874784 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.875037 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.875039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.875054 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:50.875184 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.875191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.875962 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.875965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.877357 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.877371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.878065 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.878069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.878189 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.878217 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.878826 (+ 609us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.878900 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":33,"thread_start_us":107,"threads_started":1}
I20260811 07:38:50.879185 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.874391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:36431 (local address 127.18.72.193:40643)
0811 07:38:50.874499 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.874504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.874583 (+ 79us) server_negotiation.cc:407] Connection header received
0811 07:38:50.874717 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.874723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.874784 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.874857 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:50.875177 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.875738 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.877616 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.878103 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.878212 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.878267 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.878968 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.879054 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:38:50.879652 20963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abe2639c4ce4c3ab9a723ceaa6ee24c" candidate_uuid: "1dfa28127e504428b496ba7f62c43d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1299e624f3f04d25988b35c8d78e0e4c" is_pre_election: true
I20260811 07:38:50.879696 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abe2639c4ce4c3ab9a723ceaa6ee24c" candidate_uuid: "1dfa28127e504428b496ba7f62c43d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa" is_pre_election: true
I20260811 07:38:50.880002 20963 raft_consensus.cc:2454] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dfa28127e504428b496ba7f62c43d98 in term 0.
I20260811 07:38:50.880002 20812 raft_consensus.cc:2454] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dfa28127e504428b496ba7f62c43d98 in term 0.
I20260811 07:38:50.880515 21053 leader_election.cc:302] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ac11054434b95aff06b492d2266aa, 1dfa28127e504428b496ba7f62c43d98; no voters:
I20260811 07:38:50.880738 21213 raft_consensus.cc:2781] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:50.880842 21213 raft_consensus.cc:482] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:50.880903 21213 raft_consensus.cc:3037] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:50.882017 21213 raft_consensus.cc:504] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.882387 21213 leader_election.cc:288] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [CANDIDATE]: Term 1 election: Requested vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194:40353)
I20260811 07:38:50.882710 20963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abe2639c4ce4c3ab9a723ceaa6ee24c" candidate_uuid: "1dfa28127e504428b496ba7f62c43d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1299e624f3f04d25988b35c8d78e0e4c"
I20260811 07:38:50.882711 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abe2639c4ce4c3ab9a723ceaa6ee24c" candidate_uuid: "1dfa28127e504428b496ba7f62c43d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa"
I20260811 07:38:50.882840 20963 raft_consensus.cc:3037] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:50.882858 20812 raft_consensus.cc:3037] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:50.883832 20963 raft_consensus.cc:2454] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dfa28127e504428b496ba7f62c43d98 in term 1.
I20260811 07:38:50.883836 20812 raft_consensus.cc:2454] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dfa28127e504428b496ba7f62c43d98 in term 1.
I20260811 07:38:50.884145 21053 leader_election.cc:302] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ac11054434b95aff06b492d2266aa, 1dfa28127e504428b496ba7f62c43d98; no voters:
I20260811 07:38:50.884353 21213 raft_consensus.cc:2781] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:50.884606 21213 raft_consensus.cc:686] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 1 LEADER]: Becoming Leader. State: Replica: 1dfa28127e504428b496ba7f62c43d98, State: Running, Role: LEADER
I20260811 07:38:50.884935 21213 consensus_queue.cc:237] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } }
I20260811 07:38:50.888062 20636 catalog_manager.cc:5495] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195). New cstate: current_term: 1 leader_uuid: "1dfa28127e504428b496ba7f62c43d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } health_report { overall_health: UNKNOWN } } }
Aug 11 07:38:50 dist-test-slave-zkpd krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433929, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:50.925441 21224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.917543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33815 (local address 127.0.0.1:35528)
0811 07:38:50.917899 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:50.917915 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:50.918049 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:50.918294 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:50.918296 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:50.918718 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:50.918825 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.918831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.919541 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.919543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.920103 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:50.920111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.920504 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.920506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:50.920571 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.921192 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:50.921204 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:50.922619 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:50.924070 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.924078 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.924083 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.924301 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.924554 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:50.924556 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:50.924558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:50.924634 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:50.924956 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:50.924961 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:50.925061 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 07:38:50.925151 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 07:38:50.925270 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":162,"threads_started":1}
I20260811 07:38:50.925451 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:50.917625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.18.72.254:33815)
0811 07:38:50.917754 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:50.917758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:50.918000 (+ 242us) server_negotiation.cc:407] Connection header received
0811 07:38:50.918144 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:50.918146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:50.918187 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:50.918258 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:50.918938 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.919451 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.920201 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:50.920408 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:50.920482 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:50.922744 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:50.922747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:50.922749 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:50.922769 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:50.923977 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.924394 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.924396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.924397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.924477 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:50.924711 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:50.924713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:50.924714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:50.924871 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:50.924942 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:50.925210 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 07:38:50.925330 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":38}
I20260811 07:38:50.929404 20636 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35528:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 07:38:50.930086 20636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:50.934732 20792 tablet_service.cc:1467] Processing CreateTablet for tablet acf2c5dc06094878ab8fdfe95120f884 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b1c44acbedfc42efb61172b3a3ed0ccf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:38:50.934921 21105 tablet_service.cc:1467] Processing CreateTablet for tablet acf2c5dc06094878ab8fdfe95120f884 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b1c44acbedfc42efb61172b3a3ed0ccf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:38:50.935039 20939 tablet_service.cc:1467] Processing CreateTablet for tablet acf2c5dc06094878ab8fdfe95120f884 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b1c44acbedfc42efb61172b3a3ed0ccf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:38:50.935570 20939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2c5dc06094878ab8fdfe95120f884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:50.935571 20792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2c5dc06094878ab8fdfe95120f884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:50.935640 21105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf2c5dc06094878ab8fdfe95120f884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:50.938053 21207 tablet_bootstrap.cc:492] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: Bootstrap starting.
I20260811 07:38:50.938464 21208 tablet_bootstrap.cc:492] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: Bootstrap starting.
I20260811 07:38:50.939077 21207 tablet_bootstrap.cc:654] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:50.939459 21208 tablet_bootstrap.cc:654] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:50.939908 21209 tablet_bootstrap.cc:492] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: Bootstrap starting.
I20260811 07:38:50.940546 21207 tablet_bootstrap.cc:492] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: No bootstrap required, opened a new log
I20260811 07:38:50.940634 21207 ts_tablet_manager.cc:1397] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 07:38:50.940996 21209 tablet_bootstrap.cc:654] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:50.941139 21207 raft_consensus.cc:348] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:50.941269 21207 raft_consensus.cc:374] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:50.941315 21207 raft_consensus.cc:729] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfa28127e504428b496ba7f62c43d98, State: Initialized, Role: FOLLOWER
I20260811 07:38:50.941452 21207 consensus_queue.cc:260] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:50.942068 21208 tablet_bootstrap.cc:492] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: No bootstrap required, opened a new log
I20260811 07:38:50.942140 21208 ts_tablet_manager.cc:1397] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 07:38:50.942241 21209 tablet_bootstrap.cc:492] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: No bootstrap required, opened a new log
I20260811 07:38:50.942353 21209 ts_tablet_manager.cc:1397] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:38:50.942632 21208 raft_consensus.cc:348] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:50.942724 21208 raft_consensus.cc:374] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:50.942745 21208 raft_consensus.cc:729] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075ac11054434b95aff06b492d2266aa, State: Initialized, Role: FOLLOWER
I20260811 07:38:50.942770 21209 raft_consensus.cc:348] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:50.942899 21209 raft_consensus.cc:374] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:50.942936 21209 raft_consensus.cc:729] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Initialized, Role: FOLLOWER
I20260811 07:38:50.942898 21208 consensus_queue.cc:260] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:50.943064 21209 consensus_queue.cc:260] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:50.943171 21214 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:50.943316 21214 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:50.943346 21215 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:50.943500 21215 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:50.943347 21209 ts_tablet_manager.cc:1428] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:38:50.944059 21208 ts_tablet_manager.cc:1428] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:38:50.944341 21207 ts_tablet_manager.cc:1428] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 07:38:50.944972 21213 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:50.945130 21213 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 07:38:51.025409 21183 tablet.cc:2369] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:38:51.036042 21215 raft_consensus.cc:482] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:51.036162 21215 raft_consensus.cc:504] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.037309 21215 leader_election.cc:288] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
W20260811 07:38:51.037485 21019 tablet.cc:2369] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:38:51.041631 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.037607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:43871 (local address 127.18.72.193:40643)
0811 07:38:51.037718 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:51.037723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:51.037737 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:38:51.037887 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:51.037893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:51.037946 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:51.038018 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:51.038438 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.039230 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.040469 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.041064 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.041197 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.041254 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:51.041447 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 07:38:51.041526 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:38:51.041641 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.037447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:40643 (local address 127.18.72.194:43871)
0811 07:38:51.037621 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:51.037638 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:51.037727 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:51.038175 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:51.038178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:51.038197 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:51.038326 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.038335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.039354 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.039357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.040342 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.040352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.041188 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.041192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.041307 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.041330 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 07:38:51.041426 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 07:38:51.041497 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 07:38:51.042289 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2c5dc06094878ab8fdfe95120f884" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa" is_pre_election: true
I20260811 07:38:51.042480 20812 raft_consensus.cc:2454] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 0.
I20260811 07:38:51.042838 21206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.038153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:38571 (local address 127.18.72.195:41133)
0811 07:38:51.038263 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:51.038267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:51.038460 (+ 193us) server_negotiation.cc:407] Connection header received
0811 07:38:51.038636 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:51.038639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:51.038690 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:51.038774 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:38:51.039064 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.039730 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.041261 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.041881 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.041974 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.042020 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:51.042643 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 07:38:51.042722 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:38:51.042927 20889 leader_election.cc:302] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ac11054434b95aff06b492d2266aa, 1299e624f3f04d25988b35c8d78e0e4c; no voters:
I20260811 07:38:51.043207 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.037956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:41133 (local address 127.18.72.194:38571)
0811 07:38:51.038379 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:51.038394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:51.038526 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:51.038784 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:51.038787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:51.038802 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:38:51.038953 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.038958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.039849 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.039852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.041123 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.041134 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.042387 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.042390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.042510 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.042532 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:38:51.043001 (+ 469us) client_negotiation.cc:240] Negotiation successful
0811 07:38:51.043077 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":122,"threads_started":1}
I20260811 07:38:51.043183 21215 raft_consensus.cc:2781] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:51.043278 21215 raft_consensus.cc:482] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:51.043332 21215 raft_consensus.cc:3037] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:51.043771 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2c5dc06094878ab8fdfe95120f884" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa28127e504428b496ba7f62c43d98" is_pre_election: true
I20260811 07:38:51.044036 21129 raft_consensus.cc:2454] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 0.
I20260811 07:38:51.044579 21215 raft_consensus.cc:504] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.044996 21215 leader_election.cc:288] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 election: Requested vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
I20260811 07:38:51.045324 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2c5dc06094878ab8fdfe95120f884" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa"
I20260811 07:38:51.045378 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf2c5dc06094878ab8fdfe95120f884" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa28127e504428b496ba7f62c43d98"
I20260811 07:38:51.045456 20812 raft_consensus.cc:3037] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:51.045475 21129 raft_consensus.cc:3037] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:51.046630 21129 raft_consensus.cc:2454] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 1.
I20260811 07:38:51.046630 20812 raft_consensus.cc:2454] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 1.
I20260811 07:38:51.046934 20891 leader_election.cc:302] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1299e624f3f04d25988b35c8d78e0e4c, 1dfa28127e504428b496ba7f62c43d98; no voters:
I20260811 07:38:51.047104 21215 raft_consensus.cc:2781] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:51.047360 21215 raft_consensus.cc:686] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 LEADER]: Becoming Leader. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Running, Role: LEADER
I20260811 07:38:51.047726 21215 consensus_queue.cc:237] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.050107 21229 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: TxnStatusTablet state changed. Reason: New leader 1299e624f3f04d25988b35c8d78e0e4c. Latest consensus state: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:51.050225 21229 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:38:51.050443 21231 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 07:38:51.050673 21231 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 07:38:51.051028 20636 catalog_manager.cc:5495] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194). New cstate: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } health_report { overall_health: UNKNOWN } } }
W20260811 07:38:51.065050 20862 tablet.cc:2369] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:38:51.066931 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.062954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.18.72.194:40353)
0811 07:38:51.063071 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:51.063074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:51.063187 (+ 113us) server_negotiation.cc:407] Connection header received
0811 07:38:51.063381 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:51.063384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:51.063434 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:51.063506 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:51.063772 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.064329 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.065515 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.065766 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.065856 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.066184 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.066545 (+ 361us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.066627 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:51.066761 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 07:38:51.066820 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:38:51.066955 21224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.062877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:40353 (local address 127.0.0.1:43416)
0811 07:38:51.063114 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:51.063128 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:51.063240 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:51.063550 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:51.063553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:51.063562 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:38:51.063671 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.063677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.064434 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.064438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.065388 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.065399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.065885 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.065889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.065987 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.066041 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.066655 (+ 614us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.066670 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 07:38:51.066741 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 07:38:51.066822 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 07:38:51.072165 21129 raft_consensus.cc:1261] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Refusing update from remote peer 1299e624f3f04d25988b35c8d78e0e4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:51.072185 20812 raft_consensus.cc:1261] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Refusing update from remote peer 1299e624f3f04d25988b35c8d78e0e4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:51.072775 21215 consensus_queue.cc:1035] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:51.072988 21229 consensus_queue.cc:1035] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:51.076154 21213 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: TxnStatusTablet state changed. Reason: New leader 1299e624f3f04d25988b35c8d78e0e4c. Latest consensus state: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:51.076267 21213 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:51.077186 21214 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: TxnStatusTablet state changed. Reason: New leader 1299e624f3f04d25988b35c8d78e0e4c. Latest consensus state: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:51.077312 21214 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:51.080174 21215 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:51.080281 21215 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:38:51.080413 21215 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:51.080477 21215 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 1299e624f3f04d25988b35c8d78e0e4c: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:38:51.081418 21214 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:51.081530 21214 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 075ac11054434b95aff06b492d2266aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:51.081491 21213 tablet_replica.cc:439] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } } }
I20260811 07:38:51.081588 21213 tablet_replica.cc:442] T acf2c5dc06094878ab8fdfe95120f884 P 1dfa28127e504428b496ba7f62c43d98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:38:51.082957 21237 mvcc.cc:205] Tried to move back new op lower bound from 7317233381664985088 to 7317233381574569984. Current Snapshot: MvccSnapshot[applied={T|T < 7317233381664985088}]
I20260811 07:38:51.094058 21206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.090217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.18.72.195:41133)
0811 07:38:51.090348 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:51.090351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:51.090363 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:38:51.090534 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:51.090536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:51.090585 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:51.090665 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:51.090921 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.091408 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.092625 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.092830 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.092907 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.093201 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.093587 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.093671 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:51.093827 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 07:38:51.093883 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:38:51.105305 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.101749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:40353 (local address 127.18.72.195:42643)
0811 07:38:51.101965 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:51.101979 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:51.102054 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:51.102284 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:51.102287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:51.102295 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:38:51.102418 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.102424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.103110 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.103113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.104110 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.104121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.104573 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.104577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.104671 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.104722 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.105025 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.105031 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:38:51.105098 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 07:38:51.105171 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 07:38:51.105329 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.101895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:42643 (local address 127.18.72.194:40353)
0811 07:38:51.102042 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:51.102046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:51.102073 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:38:51.102130 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:51.102134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:51.102194 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:51.102256 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:51.102527 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.103004 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.104254 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.104453 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.104527 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.104875 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.104938 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.104993 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:51.105117 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 07:38:51.105158 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 07:38:51.112568 20812 raft_consensus.cc:1261] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Refusing update from remote peer 1dfa28127e504428b496ba7f62c43d98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:51.112790 20963 raft_consensus.cc:1261] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Refusing update from remote peer 1dfa28127e504428b496ba7f62c43d98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:51.113056 21217 consensus_queue.cc:1035] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:51.113240 21213 consensus_queue.cc:1035] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:51.116482 21238 mvcc.cc:205] Tried to move back new op lower bound from 7317233381832642560 to 7317233380907438080. Current Snapshot: MvccSnapshot[applied={T|T < 7317233381832642560}]
I20260811 07:38:51.137023 21253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.132879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:41133 (local address 127.18.72.194:48255)
0811 07:38:51.133203 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:51.133217 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:51.133304 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:51.133530 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:51.133532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:51.133538 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:38:51.133641 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.133645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.134860 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.134866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.135747 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.135758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.136347 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.136350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.136426 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.136472 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.136755 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.136760 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:38:51.136808 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 07:38:51.136889 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":118,"threads_started":1}
I20260811 07:38:51.137017 21206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.132975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:48255 (local address 127.18.72.195:41133)
0811 07:38:51.133081 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:51.133085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:51.133289 (+ 204us) server_negotiation.cc:407] Connection header received
0811 07:38:51.133393 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:51.133396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:51.133444 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:51.133517 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:51.133745 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.134567 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.135859 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.136239 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.136356 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.136593 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.136674 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.136745 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:51.136820 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 07:38:51.136871 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:38:51.164191 20633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:51.164400 20633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:51.167218 20633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: Sending DeleteTablet for 3 replicas of tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c
I20260811 07:38:51.167990 20792 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:51 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45636
I20260811 07:38:51.168282 21105 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:51 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:37574
I20260811 07:38:51.168726 20939 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:51 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43400
I20260811 07:38:51.169211 21258 tablet_replica.cc:330] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: stopping tablet replica
I20260811 07:38:51.169529 21258 raft_consensus.cc:2229] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:38:51.169513 20633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 07:38:51.169881 21258 raft_consensus.cc:2258] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 07:38:51.169924 20633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:51.170504 21257 tablet_replica.cc:330] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: stopping tablet replica
I20260811 07:38:51.170866 21257 raft_consensus.cc:2229] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:51.171149 21257 raft_consensus.cc:2258] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:51.172763 21257 ts_tablet_manager.cc:1905] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:51.173022 21259 tablet_replica.cc:330] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: stopping tablet replica
I20260811 07:38:51.173300 21259 raft_consensus.cc:2229] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:51.173480 21259 raft_consensus.cc:2258] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:51.174157 21258 ts_tablet_manager.cc:1905] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:51.174749 20792 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1f567a89eb46ceb2b21ec2717608ec (DEFAULT_TABLE table=test-table [id=402c8b6e31244ce3a397d5e88a80438b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:51.175020 20792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1f567a89eb46ceb2b21ec2717608ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:51.175103 20939 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1f567a89eb46ceb2b21ec2717608ec (DEFAULT_TABLE table=test-table [id=402c8b6e31244ce3a397d5e88a80438b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:51.175367 20939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1f567a89eb46ceb2b21ec2717608ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:51.177233 21258 ts_tablet_manager.cc:1918] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:51.177327 21258 log.cc:1196] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/5abe2639c4ce4c3ab9a723ceaa6ee24c
I20260811 07:38:51.177655 21258 ts_tablet_manager.cc:1939] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1dfa28127e504428b496ba7f62c43d98: Deleting consensus metadata
I20260811 07:38:51.178009 21209 tablet_bootstrap.cc:492] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: Bootstrap starting.
I20260811 07:38:51.178221 21105 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1f567a89eb46ceb2b21ec2717608ec (DEFAULT_TABLE table=test-table [id=402c8b6e31244ce3a397d5e88a80438b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:51.178314 21208 tablet_bootstrap.cc:492] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: Bootstrap starting.
I20260811 07:38:51.178483 21105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1f567a89eb46ceb2b21ec2717608ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:51.179023 21209 tablet_bootstrap.cc:654] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:51.179278 21208 tablet_bootstrap.cc:654] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:51.180859 21207 tablet_bootstrap.cc:492] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: Bootstrap starting.
I20260811 07:38:51.181607 21208 tablet_bootstrap.cc:492] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: No bootstrap required, opened a new log
I20260811 07:38:51.181684 21208 ts_tablet_manager.cc:1397] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:38:51.181838 21207 tablet_bootstrap.cc:654] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:51.182150 21208 raft_consensus.cc:348] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.182279 21208 raft_consensus.cc:374] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:51.182309 21208 raft_consensus.cc:729] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075ac11054434b95aff06b492d2266aa, State: Initialized, Role: FOLLOWER
I20260811 07:38:51.182425 21208 consensus_queue.cc:260] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.182703 21208 ts_tablet_manager.cc:1428] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:38:51.184120 21259 ts_tablet_manager.cc:1905] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:51.186095 21209 tablet_bootstrap.cc:492] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: No bootstrap required, opened a new log
I20260811 07:38:51.186180 21209 ts_tablet_manager.cc:1397] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260811 07:38:51.186632 21209 raft_consensus.cc:348] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.186740 21209 raft_consensus.cc:374] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:51.186776 21209 raft_consensus.cc:729] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Initialized, Role: FOLLOWER
I20260811 07:38:51.186918 21209 consensus_queue.cc:260] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.187168 21209 ts_tablet_manager.cc:1428] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:38:51.188419 21257 ts_tablet_manager.cc:1918] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:51.190058 21207 tablet_bootstrap.cc:492] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: No bootstrap required, opened a new log
I20260811 07:38:51.190150 21207 ts_tablet_manager.cc:1397] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20260811 07:38:51.190600 21207 raft_consensus.cc:348] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.190708 21207 raft_consensus.cc:374] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:51.190737 21207 raft_consensus.cc:729] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfa28127e504428b496ba7f62c43d98, State: Initialized, Role: FOLLOWER
I20260811 07:38:51.190876 21207 consensus_queue.cc:260] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.191025 21259 ts_tablet_manager.cc:1918] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:51.191115 21259 log.cc:1196] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/5abe2639c4ce4c3ab9a723ceaa6ee24c
I20260811 07:38:51.191149 21207 ts_tablet_manager.cc:1428] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:38:51.191481 21259 ts_tablet_manager.cc:1939] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 1299e624f3f04d25988b35c8d78e0e4c: Deleting consensus metadata
I20260811 07:38:51.192704 20621 catalog_manager.cc:4841] TS 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133): tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c (table test-table [id=a853e25d47b24a3a8d9cef1c9bbbf7ee]) successfully deleted
I20260811 07:38:51.192898 20621 catalog_manager.cc:4841] TS 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194:40353): tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c (table test-table [id=a853e25d47b24a3a8d9cef1c9bbbf7ee]) successfully deleted
I20260811 07:38:51.193059 21257 log.cc:1196] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/5abe2639c4ce4c3ab9a723ceaa6ee24c
I20260811 07:38:51.193426 21257 ts_tablet_manager.cc:1939] T 5abe2639c4ce4c3ab9a723ceaa6ee24c P 075ac11054434b95aff06b492d2266aa: Deleting consensus metadata
I20260811 07:38:51.194666 20620 catalog_manager.cc:4841] TS 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643): tablet 5abe2639c4ce4c3ab9a723ceaa6ee24c (table test-table [id=a853e25d47b24a3a8d9cef1c9bbbf7ee]) successfully deleted
I20260811 07:38:51.246012 21229 raft_consensus.cc:482] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:51.246150 21229 raft_consensus.cc:504] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.246754 21229 leader_election.cc:288] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
I20260811 07:38:51.247195 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1f567a89eb46ceb2b21ec2717608ec" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa" is_pre_election: true
I20260811 07:38:51.247247 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1f567a89eb46ceb2b21ec2717608ec" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa28127e504428b496ba7f62c43d98" is_pre_election: true
I20260811 07:38:51.247368 20812 raft_consensus.cc:2454] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 0.
I20260811 07:38:51.247385 21129 raft_consensus.cc:2454] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 0.
I20260811 07:38:51.247694 20889 leader_election.cc:302] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ac11054434b95aff06b492d2266aa, 1299e624f3f04d25988b35c8d78e0e4c; no voters:
I20260811 07:38:51.247917 21229 raft_consensus.cc:2781] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:51.248008 21229 raft_consensus.cc:482] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:51.248064 21229 raft_consensus.cc:3037] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:51.249094 21229 raft_consensus.cc:504] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.249461 21229 leader_election.cc:288] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 election: Requested vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
I20260811 07:38:51.249825 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1f567a89eb46ceb2b21ec2717608ec" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa"
I20260811 07:38:51.249820 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1f567a89eb46ceb2b21ec2717608ec" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa28127e504428b496ba7f62c43d98"
I20260811 07:38:51.249958 20812 raft_consensus.cc:3037] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:51.249958 21129 raft_consensus.cc:3037] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:51.251065 21129 raft_consensus.cc:2454] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 1.
I20260811 07:38:51.251065 20812 raft_consensus.cc:2454] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 1.
I20260811 07:38:51.251371 20889 leader_election.cc:302] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ac11054434b95aff06b492d2266aa, 1299e624f3f04d25988b35c8d78e0e4c; no voters:
I20260811 07:38:51.251529 21229 raft_consensus.cc:2781] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:51.251619 21229 raft_consensus.cc:686] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 1 LEADER]: Becoming Leader. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Running, Role: LEADER
I20260811 07:38:51.251757 21229 consensus_queue.cc:237] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:51.253584 20633 catalog_manager.cc:5495] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194). New cstate: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } health_report { overall_health: UNKNOWN } } }
I20260811 07:38:51.271719 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.268452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.18.72.194:40353)
0811 07:38:51.268568 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:51.268571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:51.268607 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:38:51.268760 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:51.268762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:51.268802 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:51.268871 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:51.269155 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.269635 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.270742 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.270950 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.271022 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.271307 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.271373 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.271421 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:51.271544 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 07:38:51.271583 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:38:51.280961 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.277631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:48671 (local address 127.18.72.194:40353)
0811 07:38:51.277738 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:51.277741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:51.277752 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:38:51.277867 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:51.277870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:51.277909 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:51.277973 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:51.278222 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.278703 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.279878 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.280187 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.280268 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.280554 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.280618 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.280673 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:51.280794 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 07:38:51.280838 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:38:51.281104 21253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:51.277541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:40353 (local address 127.18.72.194:48671)
0811 07:38:51.277679 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:51.277693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:51.277768 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:51.278014 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:51.278018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:51.278025 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:38:51.278122 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.278128 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.278802 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.278806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.279748 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:51.279760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:51.280293 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:51.280297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:51.280385 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:51.280417 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:38:51.280699 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:38:51.280705 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:38:51.280774 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 07:38:51.280874 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 07:38:51.287181 20812 raft_consensus.cc:1261] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Refusing update from remote peer 1299e624f3f04d25988b35c8d78e0e4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:51.287302 21129 raft_consensus.cc:1261] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Refusing update from remote peer 1299e624f3f04d25988b35c8d78e0e4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:51.287751 21242 consensus_queue.cc:1035] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:51.288090 21229 consensus_queue.cc:1035] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:51.291203 21236 mvcc.cc:205] Tried to move back new op lower bound from 7317233382548860928 to 7317233382408908800. Current Snapshot: MvccSnapshot[applied={T|T < 7317233382548860928}]
W20260811 07:38:51.304741 21267 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 07:38:52.861392 20633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:52.861536 20633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:52.863787 20633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: Sending DeleteTablet for 3 replicas of tablet 6b1f567a89eb46ceb2b21ec2717608ec
I20260811 07:38:52.864394 20792 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1f567a89eb46ceb2b21ec2717608ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:52 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45636
I20260811 07:38:52.864538 20938 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1f567a89eb46ceb2b21ec2717608ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:52 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43400
I20260811 07:38:52.864799 21291 tablet_replica.cc:330] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: stopping tablet replica
I20260811 07:38:52.864893 21105 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b1f567a89eb46ceb2b21ec2717608ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:52 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:37574
I20260811 07:38:52.864980 21291 raft_consensus.cc:2229] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:52.865115 21291 raft_consensus.cc:2258] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:52.865263 21293 tablet_replica.cc:330] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: stopping tablet replica
I20260811 07:38:52.865463 21293 raft_consensus.cc:2229] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:52.865641 21293 raft_consensus.cc:2258] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:52.865837 21291 ts_tablet_manager.cc:1905] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:52.866916 21293 ts_tablet_manager.cc:1905] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:52.868481 21292 tablet_replica.cc:330] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: stopping tablet replica
I20260811 07:38:52.868705 21292 raft_consensus.cc:2229] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:38:52.869009 21291 ts_tablet_manager.cc:1918] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:38:52.869127 21291 log.cc:1196] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/6b1f567a89eb46ceb2b21ec2717608ec
I20260811 07:38:52.869246 21292 raft_consensus.cc:2258] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:52.869464 21291 ts_tablet_manager.cc:1939] T 6b1f567a89eb46ceb2b21ec2717608ec P 075ac11054434b95aff06b492d2266aa: Deleting consensus metadata
I20260811 07:38:52.870077 21292 ts_tablet_manager.cc:1905] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:52.870329 20620 catalog_manager.cc:4841] TS 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643): tablet 6b1f567a89eb46ceb2b21ec2717608ec (table test-table [id=402c8b6e31244ce3a397d5e88a80438b]) successfully deleted
I20260811 07:38:52.870544 21293 ts_tablet_manager.cc:1918] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:38:52.870631 21293 log.cc:1196] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/6b1f567a89eb46ceb2b21ec2717608ec
I20260811 07:38:52.870947 21293 ts_tablet_manager.cc:1939] T 6b1f567a89eb46ceb2b21ec2717608ec P 1dfa28127e504428b496ba7f62c43d98: Deleting consensus metadata
I20260811 07:38:52.871696 20621 catalog_manager.cc:4841] TS 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133): tablet 6b1f567a89eb46ceb2b21ec2717608ec (table test-table [id=402c8b6e31244ce3a397d5e88a80438b]) successfully deleted
I20260811 07:38:52.872678 21292 ts_tablet_manager.cc:1918] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:38:52.872745 21292 log.cc:1196] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/6b1f567a89eb46ceb2b21ec2717608ec
I20260811 07:38:52.873018 21292 ts_tablet_manager.cc:1939] T 6b1f567a89eb46ceb2b21ec2717608ec P 1299e624f3f04d25988b35c8d78e0e4c: Deleting consensus metadata
I20260811 07:38:52.873785 20621 catalog_manager.cc:4841] TS 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194:40353): tablet 6b1f567a89eb46ceb2b21ec2717608ec (table test-table [id=402c8b6e31244ce3a397d5e88a80438b]) successfully deleted
Aug 11 07:38:52 dist-test-slave-zkpd krb5kdc[20587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433932, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:38:52 dist-test-slave-zkpd krb5kdc[20587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433932, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:52.901562 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:52.886960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.18.72.254:33815)
0811 07:38:52.887272 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:52.887282 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:52.887300 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:38:52.887447 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:52.887450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:52.887500 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:52.887583 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:52.888364 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:52.888945 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:52.889749 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:52.889989 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:52.890078 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:52.898161 (+ 8083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:52.898165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:52.898167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:52.898201 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:52.899878 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:52.900386 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:52.900389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:52.900390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:52.900470 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:52.900701 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:52.900704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:52.900705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:52.900915 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:52.901019 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:52.901223 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 07:38:52.901332 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":93,"threads_started":1}
I20260811 07:38:52.904242 20633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:38:52.904711 20633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:52.909729 20792 tablet_service.cc:1467] Processing CreateTablet for tablet e8a1f40ad189492da83f4bd2b6a25763 (DEFAULT_TABLE table=test-table [id=b1590da7ab4748298f9eef204cbae9cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:52.909765 20938 tablet_service.cc:1467] Processing CreateTablet for tablet e8a1f40ad189492da83f4bd2b6a25763 (DEFAULT_TABLE table=test-table [id=b1590da7ab4748298f9eef204cbae9cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:52.909858 21105 tablet_service.cc:1467] Processing CreateTablet for tablet e8a1f40ad189492da83f4bd2b6a25763 (DEFAULT_TABLE table=test-table [id=b1590da7ab4748298f9eef204cbae9cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:52.910053 20792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a1f40ad189492da83f4bd2b6a25763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:52.910086 20938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a1f40ad189492da83f4bd2b6a25763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:52.910120 21105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a1f40ad189492da83f4bd2b6a25763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:52.912150 21302 tablet_bootstrap.cc:492] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: Bootstrap starting.
I20260811 07:38:52.912747 21305 tablet_bootstrap.cc:492] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: Bootstrap starting.
I20260811 07:38:52.913296 21304 tablet_bootstrap.cc:492] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: Bootstrap starting.
I20260811 07:38:52.913486 21302 tablet_bootstrap.cc:654] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:52.913578 21305 tablet_bootstrap.cc:654] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:52.914414 21304 tablet_bootstrap.cc:654] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:52.914884 21302 tablet_bootstrap.cc:492] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: No bootstrap required, opened a new log
I20260811 07:38:52.914983 21302 ts_tablet_manager.cc:1397] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:38:52.915210 21305 tablet_bootstrap.cc:492] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: No bootstrap required, opened a new log
I20260811 07:38:52.915297 21305 ts_tablet_manager.cc:1397] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 07:38:52.915472 21302 raft_consensus.cc:348] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:52.915580 21302 raft_consensus.cc:374] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:52.915586 21304 tablet_bootstrap.cc:492] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: No bootstrap required, opened a new log
I20260811 07:38:52.915612 21302 raft_consensus.cc:729] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075ac11054434b95aff06b492d2266aa, State: Initialized, Role: FOLLOWER
I20260811 07:38:52.915644 21304 ts_tablet_manager.cc:1397] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:38:52.915710 21305 raft_consensus.cc:348] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:52.915745 21302 consensus_queue.cc:260] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:52.915807 21305 raft_consensus.cc:374] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:52.915850 21305 raft_consensus.cc:729] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Initialized, Role: FOLLOWER
I20260811 07:38:52.916059 21305 consensus_queue.cc:260] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:52.916178 21302 ts_tablet_manager.cc:1428] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 07:38:52.916122 21304 raft_consensus.cc:348] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:52.916239 21304 raft_consensus.cc:374] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:52.916275 21304 raft_consensus.cc:729] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfa28127e504428b496ba7f62c43d98, State: Initialized, Role: FOLLOWER
I20260811 07:38:52.916327 21305 ts_tablet_manager.cc:1428] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:38:52.916421 21304 consensus_queue.cc:260] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:52.916782 21304 ts_tablet_manager.cc:1428] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:38:53.077666 21278 raft_consensus.cc:482] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:53.077816 21278 raft_consensus.cc:504] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.078380 21278 leader_election.cc:288] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
I20260811 07:38:53.078848 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a1f40ad189492da83f4bd2b6a25763" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa" is_pre_election: true
I20260811 07:38:53.078939 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a1f40ad189492da83f4bd2b6a25763" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa28127e504428b496ba7f62c43d98" is_pre_election: true
I20260811 07:38:53.079023 20812 raft_consensus.cc:2454] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 0.
I20260811 07:38:53.079141 21129 raft_consensus.cc:2454] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 0.
I20260811 07:38:53.079419 20889 leader_election.cc:302] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ac11054434b95aff06b492d2266aa, 1299e624f3f04d25988b35c8d78e0e4c; no voters:
I20260811 07:38:53.079617 21278 raft_consensus.cc:2781] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:53.079684 21278 raft_consensus.cc:482] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:53.079802 21278 raft_consensus.cc:3037] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:53.080932 21278 raft_consensus.cc:504] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.081382 21278 leader_election.cc:288] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 election: Requested vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
I20260811 07:38:53.081670 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a1f40ad189492da83f4bd2b6a25763" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa"
I20260811 07:38:53.081789 20812 raft_consensus.cc:3037] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:53.081743 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a1f40ad189492da83f4bd2b6a25763" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa28127e504428b496ba7f62c43d98"
I20260811 07:38:53.081933 21129 raft_consensus.cc:3037] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:53.082839 20812 raft_consensus.cc:2454] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 1.
I20260811 07:38:53.083067 21129 raft_consensus.cc:2454] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 1.
I20260811 07:38:53.083109 20889 leader_election.cc:302] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ac11054434b95aff06b492d2266aa, 1299e624f3f04d25988b35c8d78e0e4c; no voters:
I20260811 07:38:53.083320 21278 raft_consensus.cc:2781] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:53.083467 21278 raft_consensus.cc:686] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 LEADER]: Becoming Leader. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Running, Role: LEADER
I20260811 07:38:53.083629 21278 consensus_queue.cc:237] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.085317 20634 catalog_manager.cc:5495] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194). New cstate: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } health_report { overall_health: UNKNOWN } } }
I20260811 07:38:53.114485 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:53.111226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.18.72.194:40353)
0811 07:38:53.111512 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:53.111526 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:53.111536 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:38:53.111590 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:53.111591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:53.111631 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:53.111719 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:38:53.112027 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:53.112564 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:53.113430 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:53.113653 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:53.113765 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:53.114034 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:38:53.114108 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:38:53.114187 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:53.114279 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 07:38:53.114328 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
I20260811 07:38:53.120481 21129 raft_consensus.cc:1261] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Refusing update from remote peer 1299e624f3f04d25988b35c8d78e0e4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:53.120499 20812 raft_consensus.cc:1261] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Refusing update from remote peer 1299e624f3f04d25988b35c8d78e0e4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:53.120927 21269 consensus_queue.cc:1035] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:53.121085 21278 consensus_queue.cc:1035] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:53.150646 20636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:53.150794 20636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:53.152619 20636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: Sending DeleteTablet for 3 replicas of tablet e8a1f40ad189492da83f4bd2b6a25763
I20260811 07:38:53.153165 20938 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a1f40ad189492da83f4bd2b6a25763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:53 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43400
I20260811 07:38:53.153187 21105 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a1f40ad189492da83f4bd2b6a25763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:53 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:37574
I20260811 07:38:53.153355 21293 tablet_replica.cc:330] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: stopping tablet replica
I20260811 07:38:53.153357 21292 tablet_replica.cc:330] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: stopping tablet replica
I20260811 07:38:53.153400 20792 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a1f40ad189492da83f4bd2b6a25763 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:53 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45636
I20260811 07:38:53.153468 21292 raft_consensus.cc:2229] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:38:53.153468 21293 raft_consensus.cc:2229] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:53.153568 21293 raft_consensus.cc:2258] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:53.153601 21291 tablet_replica.cc:330] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: stopping tablet replica
I20260811 07:38:53.153636 21292 raft_consensus.cc:2258] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:53.153759 21291 raft_consensus.cc:2229] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:53.153879 21291 raft_consensus.cc:2258] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:53.154347 21293 ts_tablet_manager.cc:1905] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:53.154479 21292 ts_tablet_manager.cc:1905] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:53.154631 20636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 07:38:53.154866 21291 ts_tablet_manager.cc:1905] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 07:38:53.155004 20636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:38:53.157056 21293 ts_tablet_manager.cc:1918] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:53.157112 21293 log.cc:1196] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/e8a1f40ad189492da83f4bd2b6a25763
I20260811 07:38:53.157202 21292 ts_tablet_manager.cc:1918] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:53.157281 21292 log.cc:1196] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/e8a1f40ad189492da83f4bd2b6a25763
I20260811 07:38:53.157330 21293 ts_tablet_manager.cc:1939] T e8a1f40ad189492da83f4bd2b6a25763 P 1dfa28127e504428b496ba7f62c43d98: Deleting consensus metadata
I20260811 07:38:53.157408 21291 ts_tablet_manager.cc:1918] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:38:53.157477 21291 log.cc:1196] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/e8a1f40ad189492da83f4bd2b6a25763
I20260811 07:38:53.157574 21292 ts_tablet_manager.cc:1939] T e8a1f40ad189492da83f4bd2b6a25763 P 1299e624f3f04d25988b35c8d78e0e4c: Deleting consensus metadata
I20260811 07:38:53.157738 21291 ts_tablet_manager.cc:1939] T e8a1f40ad189492da83f4bd2b6a25763 P 075ac11054434b95aff06b492d2266aa: Deleting consensus metadata
I20260811 07:38:53.158083 20621 catalog_manager.cc:4841] TS 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133): tablet e8a1f40ad189492da83f4bd2b6a25763 (table test-table [id=b1590da7ab4748298f9eef204cbae9cb]) successfully deleted
I20260811 07:38:53.158259 20621 catalog_manager.cc:4841] TS 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194:40353): tablet e8a1f40ad189492da83f4bd2b6a25763 (table test-table [id=b1590da7ab4748298f9eef204cbae9cb]) successfully deleted
I20260811 07:38:53.158306 20620 catalog_manager.cc:4841] TS 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643): tablet e8a1f40ad189492da83f4bd2b6a25763 (table test-table [id=b1590da7ab4748298f9eef204cbae9cb]) successfully deleted
I20260811 07:38:53.162372 20792 tablet_service.cc:1467] Processing CreateTablet for tablet 7efe6045653f42a7b9b5483d6e7e3d97 (DEFAULT_TABLE table=test-table [id=4bfeaf515ac940b9a7658187e68073f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:53.162533 20938 tablet_service.cc:1467] Processing CreateTablet for tablet 7efe6045653f42a7b9b5483d6e7e3d97 (DEFAULT_TABLE table=test-table [id=4bfeaf515ac940b9a7658187e68073f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:53.162581 21105 tablet_service.cc:1467] Processing CreateTablet for tablet 7efe6045653f42a7b9b5483d6e7e3d97 (DEFAULT_TABLE table=test-table [id=4bfeaf515ac940b9a7658187e68073f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:38:53.162652 20792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efe6045653f42a7b9b5483d6e7e3d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:53.162811 21105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efe6045653f42a7b9b5483d6e7e3d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:53.162875 20938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7efe6045653f42a7b9b5483d6e7e3d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:53.164904 21305 tablet_bootstrap.cc:492] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: Bootstrap starting.
I20260811 07:38:53.165019 21304 tablet_bootstrap.cc:492] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: Bootstrap starting.
I20260811 07:38:53.165019 21302 tablet_bootstrap.cc:492] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa: Bootstrap starting.
I20260811 07:38:53.165709 21305 tablet_bootstrap.cc:654] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:53.165824 21304 tablet_bootstrap.cc:654] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:53.165881 21302 tablet_bootstrap.cc:654] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:53.166792 21305 tablet_bootstrap.cc:492] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: No bootstrap required, opened a new log
I20260811 07:38:53.166898 21305 ts_tablet_manager.cc:1397] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:38:53.167109 21302 tablet_bootstrap.cc:492] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa: No bootstrap required, opened a new log
I20260811 07:38:53.167179 21302 ts_tablet_manager.cc:1397] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 07:38:53.167310 21305 raft_consensus.cc:348] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.167431 21305 raft_consensus.cc:374] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:53.167475 21305 raft_consensus.cc:729] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Initialized, Role: FOLLOWER
I20260811 07:38:53.167606 21305 consensus_queue.cc:260] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.167618 21302 raft_consensus.cc:348] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.167707 21302 raft_consensus.cc:374] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:53.167744 21302 raft_consensus.cc:729] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075ac11054434b95aff06b492d2266aa, State: Initialized, Role: FOLLOWER
I20260811 07:38:53.167922 21305 ts_tablet_manager.cc:1428] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:38:53.167872 21302 consensus_queue.cc:260] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.168140 21302 ts_tablet_manager.cc:1428] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 07:38:53.168679 21304 tablet_bootstrap.cc:492] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: No bootstrap required, opened a new log
I20260811 07:38:53.168769 21304 ts_tablet_manager.cc:1397] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 07:38:53.169102 21304 raft_consensus.cc:348] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.169179 21304 raft_consensus.cc:374] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:53.169200 21304 raft_consensus.cc:729] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfa28127e504428b496ba7f62c43d98, State: Initialized, Role: FOLLOWER
I20260811 07:38:53.169327 21304 consensus_queue.cc:260] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.169591 21304 ts_tablet_manager.cc:1428] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 07:38:53.362586 21272 raft_consensus.cc:482] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:38:53.362752 21272 raft_consensus.cc:504] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.363274 21272 leader_election.cc:288] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
I20260811 07:38:53.363842 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efe6045653f42a7b9b5483d6e7e3d97" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa28127e504428b496ba7f62c43d98" is_pre_election: true
I20260811 07:38:53.363844 20811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efe6045653f42a7b9b5483d6e7e3d97" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa" is_pre_election: true
I20260811 07:38:53.364034 21129 raft_consensus.cc:2454] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 0.
I20260811 07:38:53.364056 20811 raft_consensus.cc:2454] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 0.
I20260811 07:38:53.364403 20891 leader_election.cc:302] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1299e624f3f04d25988b35c8d78e0e4c, 1dfa28127e504428b496ba7f62c43d98; no voters:
I20260811 07:38:53.364657 21272 raft_consensus.cc:2781] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:38:53.364719 21272 raft_consensus.cc:482] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:38:53.364768 21272 raft_consensus.cc:3037] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:53.365959 21272 raft_consensus.cc:504] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.366317 21272 leader_election.cc:288] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 election: Requested vote from peers 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643), 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133)
I20260811 07:38:53.366701 20811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efe6045653f42a7b9b5483d6e7e3d97" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075ac11054434b95aff06b492d2266aa"
I20260811 07:38:53.366734 21129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7efe6045653f42a7b9b5483d6e7e3d97" candidate_uuid: "1299e624f3f04d25988b35c8d78e0e4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa28127e504428b496ba7f62c43d98"
I20260811 07:38:53.366845 20811 raft_consensus.cc:3037] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:53.366858 21129 raft_consensus.cc:3037] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:53.367828 20811 raft_consensus.cc:2454] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 1.
I20260811 07:38:53.367823 21129 raft_consensus.cc:2454] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299e624f3f04d25988b35c8d78e0e4c in term 1.
I20260811 07:38:53.368173 20889 leader_election.cc:302] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ac11054434b95aff06b492d2266aa, 1299e624f3f04d25988b35c8d78e0e4c; no voters:
I20260811 07:38:53.368360 21272 raft_consensus.cc:2781] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:53.368453 21272 raft_consensus.cc:686] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 LEADER]: Becoming Leader. State: Replica: 1299e624f3f04d25988b35c8d78e0e4c, State: Running, Role: LEADER
I20260811 07:38:53.368598 21272 consensus_queue.cc:237] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } }
I20260811 07:38:53.370472 20633 catalog_manager.cc:5495] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194). New cstate: current_term: 1 leader_uuid: "1299e624f3f04d25988b35c8d78e0e4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1299e624f3f04d25988b35c8d78e0e4c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 40353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 } health_report { overall_health: UNKNOWN } } }
I20260811 07:38:53.414597 21129 raft_consensus.cc:1261] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Refusing update from remote peer 1299e624f3f04d25988b35c8d78e0e4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:53.414973 20811 raft_consensus.cc:1261] T 7efe6045653f42a7b9b5483d6e7e3d97 P 075ac11054434b95aff06b492d2266aa [term 1 FOLLOWER]: Refusing update from remote peer 1299e624f3f04d25988b35c8d78e0e4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:38:53.415022 21272 consensus_queue.cc:1035] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dfa28127e504428b496ba7f62c43d98" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:38:53.415344 21272 consensus_queue.cc:1035] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "075ac11054434b95aff06b492d2266aa" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 07:38:53.428026 21270 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 07:38:54.974658 20633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:54.974807 20633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:38:54.977052 20633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48c36784ecdb4b8897db0e621f97c1b6: Sending DeleteTablet for 3 replicas of tablet 7efe6045653f42a7b9b5483d6e7e3d97
I20260811 07:38:54.977591 20938 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efe6045653f42a7b9b5483d6e7e3d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:54 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43400
I20260811 07:38:54.977581 20792 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efe6045653f42a7b9b5483d6e7e3d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:54 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45636
I20260811 07:38:54.977689 21105 tablet_service.cc:1514] Processing DeleteTablet for tablet 7efe6045653f42a7b9b5483d6e7e3d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:38:54 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:37574
I20260811 07:38:54.977936 21349 tablet_replica.cc:330] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: stopping tablet replica
I20260811 07:38:54.978030 21351 tablet_replica.cc:330] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: stopping tablet replica
I20260811 07:38:54.978030 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 20719
I20260811 07:38:54.981769 21351 raft_consensus.cc:2229] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:38:54.981916 21351 raft_consensus.cc:2258] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:54.982237 21349 raft_consensus.cc:2229] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:38:54.982491 21349 raft_consensus.cc:2258] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:38:54.982618 21351 ts_tablet_manager.cc:1905] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:54.983203 21349 ts_tablet_manager.cc:1905] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:38:54.985611 21351 ts_tablet_manager.cc:1918] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:38:54.985697 21351 log.cc:1196] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/7efe6045653f42a7b9b5483d6e7e3d97
I20260811 07:38:54.986060 21351 ts_tablet_manager.cc:1939] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1dfa28127e504428b496ba7f62c43d98: Deleting consensus metadata
I20260811 07:38:54.986120 21349 ts_tablet_manager.cc:1918] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 07:38:54.986198 21349 log.cc:1196] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/7efe6045653f42a7b9b5483d6e7e3d97
I20260811 07:38:54.986492 21349 ts_tablet_manager.cc:1939] T 7efe6045653f42a7b9b5483d6e7e3d97 P 1299e624f3f04d25988b35c8d78e0e4c: Deleting consensus metadata
I20260811 07:38:54.986964 20621 catalog_manager.cc:4841] TS 1dfa28127e504428b496ba7f62c43d98 (127.18.72.195:41133): tablet 7efe6045653f42a7b9b5483d6e7e3d97 (table test-table [id=4bfeaf515ac940b9a7658187e68073f3]) successfully deleted
I20260811 07:38:54.987149 20621 catalog_manager.cc:4841] TS 1299e624f3f04d25988b35c8d78e0e4c (127.18.72.194:40353): tablet 7efe6045653f42a7b9b5483d6e7e3d97 (table test-table [id=4bfeaf515ac940b9a7658187e68073f3]) successfully deleted
W20260811 07:38:54.991184 20620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:40643 (error 108)
I20260811 07:38:54.992002 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 20870
I20260811 07:38:54.995114 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:54.991962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:40643 (local address 127.0.0.1:45652)
0811 07:38:54.994847 (+ 2885us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:54.994947 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:40643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2781,"thread_start_us":98,"threads_started":1}
W20260811 07:38:54.995260 20620 catalog_manager.cc:4584] TS 075ac11054434b95aff06b492d2266aa (127.18.72.193:40643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7efe6045653f42a7b9b5483d6e7e3d97: Network error: Client connection negotiation failed: client connection to 127.18.72.193:40643: connect: Connection refused (error 111)
I20260811 07:38:55.002333 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 21029
I20260811 07:38:55.011737 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 20601
2026-08-11T07:38:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6389 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:38:55 dist-test-slave-zkpd krb5kdc[21356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:38:55 dist-test-slave-zkpd krb5kdc[21356](info): set up 2 sockets
Aug 11 07:38:55 dist-test-slave-zkpd krb5kdc[21356](info): commencing operation
krb5kdc: starting...
W20260811 07:38:56.089185 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:38:56 dist-test-slave-zkpd krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:38:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:38:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:38:56.214694 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:35729
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:35729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:56.329576 21369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:56.330030 21369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:56.330137 21369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:56.335013 21369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:38:56.335093 21369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:56.335115 21369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:56.335132 21369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:38:56.335148 21369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:38:56.340978 21369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:35729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:35729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:56.342114 21369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:56.343216 21369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:56.351929 21375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:56.351974 21377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:56.352016 21374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:56.352226 21369 server_base.cc:1034] running on GCE node
I20260811 07:38:56.352793 21369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:56.353852 21369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:56.355015 21369 hybrid_clock.cc:648] HybridClock initialized: now 1786433936354997 us; error 33 us; skew 500 ppm
Aug 11 07:38:56 dist-test-slave-zkpd krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:56.357762 21369 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:38:56.358569 21369 webserver.cc:460] Webserver started at http://127.18.72.254:34351/ using document root <none> and password file <none>
I20260811 07:38:56.359099 21369 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:56.359158 21369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:56.359328 21369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:56.361402 21369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "bcdbcddeca90496ca5bb11bfcf7e240b"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "d520d7b51fd1f9dca7242bafaaa481de"
server_key_iv: "f8ce753531108a41221e61d742674e55"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.361922 21369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "bcdbcddeca90496ca5bb11bfcf7e240b"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "d520d7b51fd1f9dca7242bafaaa481de"
server_key_iv: "f8ce753531108a41221e61d742674e55"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.365650 21369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:38:56.368064 21384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:56.369313 21369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:38:56.369442 21369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "bcdbcddeca90496ca5bb11bfcf7e240b"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "d520d7b51fd1f9dca7242bafaaa481de"
server_key_iv: "f8ce753531108a41221e61d742674e55"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.369549 21369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:56.380844 21369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:56.383642 21369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:56.383852 21369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:56.391426 21369 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:35729
I20260811 07:38:56.391436 21443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:35729 every 8 connection(s)
I20260811 07:38:56.392423 21369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:38:56.395323 21444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:56.400933 21444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b: Bootstrap starting.
I20260811 07:38:56.401237 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 21369
I20260811 07:38:56.401347 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:38:56.401623 18723 external_mini_cluster.cc:1428] Setting key ff0afd9f35fbd3f68d0e0185808eabf4
I20260811 07:38:56.403733 21444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:56.404558 21444 log.cc:824] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:56.406795 21444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b: No bootstrap required, opened a new log
I20260811 07:38:56.411345 21444 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdbcddeca90496ca5bb11bfcf7e240b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35729 } }
I20260811 07:38:56.411553 21444 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:56.411586 21444 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcdbcddeca90496ca5bb11bfcf7e240b, State: Initialized, Role: FOLLOWER
Aug 11 07:38:56 dist-test-slave-zkpd krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:56.412091 21444 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdbcddeca90496ca5bb11bfcf7e240b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35729 } }
I20260811 07:38:56.412213 21444 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:38:56.412289 21444 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:38:56.412410 21444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:56.413679 21444 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdbcddeca90496ca5bb11bfcf7e240b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35729 } }
I20260811 07:38:56.414084 21444 leader_election.cc:302] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcdbcddeca90496ca5bb11bfcf7e240b; no voters:
I20260811 07:38:56.414466 21444 leader_election.cc:288] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:38:56.414575 21449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:56.414815 21449 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [term 1 LEADER]: Becoming Leader. State: Replica: bcdbcddeca90496ca5bb11bfcf7e240b, State: Running, Role: LEADER
I20260811 07:38:56.415184 21449 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdbcddeca90496ca5bb11bfcf7e240b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35729 } }
I20260811 07:38:56.415872 21444 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:38:56.417109 21452 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcdbcddeca90496ca5bb11bfcf7e240b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdbcddeca90496ca5bb11bfcf7e240b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35729 } } }
I20260811 07:38:56.417244 21452 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:56.417682 21461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:38:56.417984 21454 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcdbcddeca90496ca5bb11bfcf7e240b. Latest consensus state: current_term: 1 leader_uuid: "bcdbcddeca90496ca5bb11bfcf7e240b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdbcddeca90496ca5bb11bfcf7e240b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35729 } } }
I20260811 07:38:56.418076 21454 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:56.419075 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:56.402975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.18.72.254:35729)
0811 07:38:56.403469 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:56.403500 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:56.403546 (+ 46us) server_negotiation.cc:407] Connection header received
0811 07:38:56.404452 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:56.404473 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:56.404957 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:56.405405 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:56.406252 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.407416 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.408642 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.409177 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.409303 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:56.412484 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:56.412492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:56.412509 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:56.412559 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:56.414604 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:56.415153 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:56.415158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:56.415164 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:56.415359 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:56.415686 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:56.415690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:56.415691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:56.416243 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:56.416473 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:56.417583 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0811 07:38:56.417939 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":131,"threads_started":1}
I20260811 07:38:56.421245 21461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:38:56.427325 21461 catalog_manager.cc:1347] Generated new cluster ID: 0dd4b2e983574e1e823fe89a2351edba
I20260811 07:38:56.427402 21461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:38:56.449376 21461 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:38:56.450412 21461 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:38:56.460700 21461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcdbcddeca90496ca5bb11bfcf7e240b: Generated new TSK 0
I20260811 07:38:56.461539 21461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:38:56.495256 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35729
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:56.620164 21474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:56.620421 21474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:56.620463 21474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:56.624342 21474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:56.624399 21474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:56.624455 21474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:38:56.629565 21474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:56.630590 21474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:56.631645 21474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:56.641731 21481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:56.641711 21480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:56.641718 21483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:56.641793 21474 server_base.cc:1034] running on GCE node
I20260811 07:38:56.642356 21474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:56.643031 21474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:56.644234 21474 hybrid_clock.cc:648] HybridClock initialized: now 1786433936644222 us; error 30 us; skew 500 ppm
Aug 11 07:38:56 dist-test-slave-zkpd krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:56.647047 21474 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:38:56.648007 21474 webserver.cc:460] Webserver started at http://127.18.72.193:40915/ using document root <none> and password file <none>
I20260811 07:38:56.648602 21474 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:56.648663 21474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:56.648833 21474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:56.650866 21474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "989314ad6cbb4fecb2aae0e41a5770c6"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "f01596eefdd1bdbc2aeec69e723e9acc"
server_key_iv: "6c5393cf2d3f4b26774ceeb742be059f"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.651422 21474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "989314ad6cbb4fecb2aae0e41a5770c6"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "f01596eefdd1bdbc2aeec69e723e9acc"
server_key_iv: "6c5393cf2d3f4b26774ceeb742be059f"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.655144 21474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:38:56.657660 21490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:56.658874 21474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:38:56.658972 21474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "989314ad6cbb4fecb2aae0e41a5770c6"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "f01596eefdd1bdbc2aeec69e723e9acc"
server_key_iv: "6c5393cf2d3f4b26774ceeb742be059f"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.659054 21474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:56.686051 21474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:56.689404 21474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:56.689574 21474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:56.690155 21474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:56.691001 21474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:56.691051 21474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:56.691092 21474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:56.691114 21474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:56.700464 21474 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:35715
I20260811 07:38:56.700489 21617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:35715 every 8 connection(s)
I20260811 07:38:56.701403 21474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:38:56.703130 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 21474
I20260811 07:38:56.703266 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:38:56.703568 18723 external_mini_cluster.cc:1428] Setting key da3fbcc4d7fb979600c4ecb45814b0e6
Aug 11 07:38:56 dist-test-slave-zkpd krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
I20260811 07:38:56.716622 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:56.703308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:54017 (local address 127.18.72.254:35729)
0811 07:38:56.703457 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:56.703461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:56.704571 (+ 1110us) server_negotiation.cc:407] Connection header received
0811 07:38:56.706417 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:56.706420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:56.706463 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:56.706535 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:56.708059 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.708591 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.709794 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.710245 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.710371 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:56.712906 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:56.712909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:56.712911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:56.712933 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:56.714447 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:56.714968 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:56.714972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:56.714973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:56.715046 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:56.715399 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:56.715402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:56.715404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:56.715583 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:56.715690 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:56.716322 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 07:38:56.716445 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:38:56.717494 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:56.703803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:35729 (local address 127.18.72.193:54017)
0811 07:38:56.704425 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:56.704471 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:56.706168 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:56.706726 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:56.706738 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:56.707278 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:56.707864 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:56.707878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.708743 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.708764 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:56.709652 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:56.709663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.710415 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.710417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:56.710559 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:56.711217 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:56.711233 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:56.712726 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:56.714565 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:56.714571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:56.714585 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:56.714850 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:56.715146 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:56.715149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:56.715151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:56.715295 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:56.715700 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:56.715709 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:56.716058 (+ 349us) client_negotiation.cc:769] Sending connection context
0811 07:38:56.716256 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 07:38:56.716583 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":154,"threads_started":1}
I20260811 07:38:56.718816 21618 heartbeater.cc:347] Connected to a master server at 127.18.72.254:35729
I20260811 07:38:56.719091 21618 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:56.719705 21618 heartbeater.cc:511] Master 127.18.72.254:35729 requested a full tablet report, sending...
I20260811 07:38:56.721200 21404 ts_manager.cc:194] Registered new tserver with Master: 989314ad6cbb4fecb2aae0e41a5770c6 (127.18.72.193:35715)
I20260811 07:38:56.722328 21404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:54017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:38:56.734218 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35729
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:56.850162 21623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:56.850420 21623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:56.850495 21623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:56.854499 21623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:56.854558 21623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:56.854671 21623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:38:56.860529 21623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:56.861567 21623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:56.862666 21623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:56.872545 21630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:56.872568 21623 server_base.cc:1034] running on GCE node
W20260811 07:38:56.872568 21632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:56.872781 21629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:56.873248 21623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:56.873943 21623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:56.875104 21623 hybrid_clock.cc:648] HybridClock initialized: now 1786433936875069 us; error 45 us; skew 500 ppm
Aug 11 07:38:56 dist-test-slave-zkpd krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:56.877929 21623 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:38:56.878798 21623 webserver.cc:460] Webserver started at http://127.18.72.194:33631/ using document root <none> and password file <none>
I20260811 07:38:56.879376 21623 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:56.879469 21623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:56.879710 21623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:56.881837 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "d24cbbb71a09475fb2e431ded88902d8"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "f2829101e9296dbbdeb0818b576fb74e"
server_key_iv: "b61aa17c73d5da076556b37464fa15ee"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.882421 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "d24cbbb71a09475fb2e431ded88902d8"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "f2829101e9296dbbdeb0818b576fb74e"
server_key_iv: "b61aa17c73d5da076556b37464fa15ee"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.886214 21623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 07:38:56.888630 21641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:56.889868 21623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:38:56.890003 21623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "d24cbbb71a09475fb2e431ded88902d8"
format_stamp: "Formatted at 2026-08-11 07:38:56 on dist-test-slave-zkpd"
server_key: "f2829101e9296dbbdeb0818b576fb74e"
server_key_iv: "b61aa17c73d5da076556b37464fa15ee"
server_key_version: "encryptionkey@0"
I20260811 07:38:56.890115 21623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:56.919121 21623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:56.922369 21623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:56.922593 21623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:56.923236 21623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:56.924175 21623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:56.924224 21623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:56.924291 21623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:56.924329 21623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:56.933167 21623 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:43965
I20260811 07:38:56.933193 21764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:43965 every 8 connection(s)
I20260811 07:38:56.934121 21623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:38:56.941354 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 21623
I20260811 07:38:56.941486 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:38:56.941774 18723 external_mini_cluster.cc:1428] Setting key d8a8bb2bc3034791f49aaba17d459d64
Aug 11 07:38:56 dist-test-slave-zkpd krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:56.949618 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:56.936026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:36349 (local address 127.18.72.254:35729)
0811 07:38:56.936161 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:56.936165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:56.937043 (+ 878us) server_negotiation.cc:407] Connection header received
0811 07:38:56.938397 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:56.938401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:56.938462 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:56.938571 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:56.939750 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.940420 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.941669 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.941945 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.942070 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:56.945179 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:56.945185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:56.945191 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:56.945227 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:56.947100 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:56.947737 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:56.947739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:56.947740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:56.947815 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:56.948237 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:56.948239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:56.948240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:56.948420 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:56.948497 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:56.949256 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 07:38:56.949415 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:38:56.951459 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:56.936434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:35729 (local address 127.18.72.194:36349)
0811 07:38:56.936892 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:56.936934 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:56.938180 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:56.938751 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:56.938758 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:56.939122 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:56.939603 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:56.939616 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.940562 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.940567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:56.941461 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:56.941499 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:56.942112 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:56.942117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:56.942337 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:56.943064 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:56.943086 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:56.944976 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:56.947223 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:56.947232 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:56.947253 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:56.947638 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:56.947934 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:56.947939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:56.947944 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:56.948139 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:56.948528 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:56.948538 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:56.948922 (+ 384us) client_negotiation.cc:769] Sending connection context
0811 07:38:56.949809 (+ 887us) client_negotiation.cc:240] Negotiation successful
0811 07:38:56.950218 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":116,"threads_started":1}
I20260811 07:38:56.952687 21765 heartbeater.cc:347] Connected to a master server at 127.18.72.254:35729
I20260811 07:38:56.952967 21765 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:56.953486 21765 heartbeater.cc:511] Master 127.18.72.254:35729 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:38:56.954605 21404 ts_manager.cc:194] Registered new tserver with Master: d24cbbb71a09475fb2e431ded88902d8 (127.18.72.194:43965)
I20260811 07:38:56.955214 21404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:36349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:38:56.971858 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35729
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:57.098964 21771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:57.099258 21771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:57.099330 21771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:57.103880 21771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:57.103987 21771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:57.104079 21771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:38:57.109553 21771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:57.110708 21771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:57.111760 21771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:57.121666 21779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:57.121662 21777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:57.121691 21776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:57.121735 21771 server_base.cc:1034] running on GCE node
I20260811 07:38:57.122418 21771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:57.123039 21771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:57.124222 21771 hybrid_clock.cc:648] HybridClock initialized: now 1786433937124210 us; error 31 us; skew 500 ppm
Aug 11 07:38:57 dist-test-slave-zkpd krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433937, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:57.126909 21771 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:38:57.127708 21771 webserver.cc:460] Webserver started at http://127.18.72.195:39633/ using document root <none> and password file <none>
I20260811 07:38:57.128309 21771 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:57.128371 21771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:57.128607 21771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:57.130604 21771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "27875c7d9a984d12ae599e7871a89202"
format_stamp: "Formatted at 2026-08-11 07:38:57 on dist-test-slave-zkpd"
server_key: "3b43508dfa284688f59415f106555d76"
server_key_iv: "36f87086f8f618b7de37f9dbf966f4be"
server_key_version: "encryptionkey@0"
I20260811 07:38:57.131178 21771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "27875c7d9a984d12ae599e7871a89202"
format_stamp: "Formatted at 2026-08-11 07:38:57 on dist-test-slave-zkpd"
server_key: "3b43508dfa284688f59415f106555d76"
server_key_iv: "36f87086f8f618b7de37f9dbf966f4be"
server_key_version: "encryptionkey@0"
I20260811 07:38:57.135064 21771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:38:57.137636 21788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:57.139106 21771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 07:38:57.139251 21771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "27875c7d9a984d12ae599e7871a89202"
format_stamp: "Formatted at 2026-08-11 07:38:57 on dist-test-slave-zkpd"
server_key: "3b43508dfa284688f59415f106555d76"
server_key_iv: "36f87086f8f618b7de37f9dbf966f4be"
server_key_version: "encryptionkey@0"
I20260811 07:38:57.139370 21771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:57.154031 21771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:57.156898 21771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:57.157104 21771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:57.157708 21771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:38:57.158593 21771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:38:57.158664 21771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:57.158735 21771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:38:57.158782 21771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:57.168429 21771 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:37649
I20260811 07:38:57.168452 21913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:37649 every 8 connection(s)
I20260811 07:38:57.169375 21771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
Aug 11 07:38:57 dist-test-slave-zkpd krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433937, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:57.179419 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 21771
I20260811 07:38:57.179585 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:38:57.179924 18723 external_mini_cluster.cc:1428] Setting key 11697aa7d0026ca2dfbe3fdb2c7f775c
I20260811 07:38:57.183120 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:57.171229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:54241 (local address 127.18.72.254:35729)
0811 07:38:57.171373 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:57.171377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:57.172264 (+ 887us) server_negotiation.cc:407] Connection header received
0811 07:38:57.173596 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:57.173599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:57.173645 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:57.173742 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:57.174967 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:57.175493 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:57.176696 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:57.176913 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:57.176996 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:57.179414 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:57.179418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:57.179420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:57.179443 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:57.181080 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:57.181609 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:57.181613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:57.181614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:57.181696 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:57.182040 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:57.182045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:57.182047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:57.182233 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:57.182347 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:57.182849 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 07:38:57.182964 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:38:57.184118 21917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:57.171618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:35729 (local address 127.18.72.195:54241)
0811 07:38:57.172097 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 07:38:57.172139 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:38:57.173402 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:38:57.173917 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:38:57.173923 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:38:57.174338 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:38:57.174836 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:57.174849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:57.175607 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:57.175609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:57.176585 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:38:57.176595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:38:57.177049 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:38:57.177052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:38:57.177191 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:57.177783 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:38:57.177800 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:38:57.179231 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:38:57.181204 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:57.181211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:57.181227 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:57.181493 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:57.181788 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:38:57.181795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:38:57.181798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:38:57.181942 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:38:57.182345 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:38:57.182353 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:38:57.182609 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 07:38:57.182805 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 07:38:57.183139 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":110,"threads_started":1}
I20260811 07:38:57.185405 21914 heartbeater.cc:347] Connected to a master server at 127.18.72.254:35729
I20260811 07:38:57.185670 21914 heartbeater.cc:464] Registering TS with master...
I20260811 07:38:57.186197 21914 heartbeater.cc:511] Master 127.18.72.254:35729 requested a full tablet report, sending...
I20260811 07:38:57.187273 21404 ts_manager.cc:194] Registered new tserver with Master: 27875c7d9a984d12ae599e7871a89202 (127.18.72.195:37649)
I20260811 07:38:57.187832 21404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:54241
I20260811 07:38:57.193897 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 07:38:57.210184 21447 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:38:57.208543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33998 (local address 127.18.72.254:35729)
0811 07:38:57.208730 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:57.208735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:57.208883 (+ 148us) server_negotiation.cc:407] Connection header received
0811 07:38:57.209062 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:57.209065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:57.209118 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:57.209210 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:57.210014 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33998: BlockingRecv error: recv got EOF from 127.0.0.1:33998 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 07:38:57.211133 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 21474
I20260811 07:38:57.221124 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 21623
I20260811 07:38:57.230216 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 21771
I20260811 07:38:57.238986 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 21369
2026-08-11T07:38:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2226 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:38:57 dist-test-slave-zkpd krb5kdc[21929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:38:57 dist-test-slave-zkpd krb5kdc[21929](info): set up 2 sockets
Aug 11 07:38:57 dist-test-slave-zkpd krb5kdc[21929](info): commencing operation
krb5kdc: starting...
W20260811 07:38:58.324412 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:38:58 dist-test-slave-zkpd krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:38:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:38:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:38:58.450376 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:44841
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:44841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:38:58.573890 21947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:38:58.574151 21947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:38:58.574193 21947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:38:58.578270 21947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:38:58.578346 21947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:38:58.578365 21947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:38:58.578382 21947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:38:58.578397 21947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:38:58.583622 21947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:44841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:44841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:38:58.584709 21947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:38:58.585763 21947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:38:58.594425 21957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:58.594451 21954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:38:58.594425 21955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:38:58.594794 21947 server_base.cc:1034] running on GCE node
I20260811 07:38:58.595413 21947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:38:58.596673 21947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:38:58.597849 21947 hybrid_clock.cc:648] HybridClock initialized: now 1786433938597812 us; error 50 us; skew 500 ppm
Aug 11 07:38:58 dist-test-slave-zkpd krb5kdc[21929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433938, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:38:58.600826 21947 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:38:58.601742 21947 webserver.cc:460] Webserver started at http://127.18.72.254:44995/ using document root <none> and password file <none>
I20260811 07:38:58.602809 21947 fs_manager.cc:359] Metadata directory not provided
I20260811 07:38:58.602895 21947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:38:58.603135 21947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:38:58.605355 21947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "98a627119ae94ce1ace8e12d3e3e665e"
format_stamp: "Formatted at 2026-08-11 07:38:58 on dist-test-slave-zkpd"
server_key: "2fb34bdbc619c0c5965cbfe4c59becf1"
server_key_iv: "400485fc38542205624ffd99f86078e3"
server_key_version: "encryptionkey@0"
I20260811 07:38:58.605958 21947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "98a627119ae94ce1ace8e12d3e3e665e"
format_stamp: "Formatted at 2026-08-11 07:38:58 on dist-test-slave-zkpd"
server_key: "2fb34bdbc619c0c5965cbfe4c59becf1"
server_key_iv: "400485fc38542205624ffd99f86078e3"
server_key_version: "encryptionkey@0"
I20260811 07:38:58.609863 21947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 07:38:58.612337 21965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:38:58.613699 21947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:38:58.613809 21947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "98a627119ae94ce1ace8e12d3e3e665e"
format_stamp: "Formatted at 2026-08-11 07:38:58 on dist-test-slave-zkpd"
server_key: "2fb34bdbc619c0c5965cbfe4c59becf1"
server_key_iv: "400485fc38542205624ffd99f86078e3"
server_key_version: "encryptionkey@0"
I20260811 07:38:58.613969 21947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:38:58.637356 21947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:38:58.640524 21947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:38:58.640746 21947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:38:58.648460 21947 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:44841
I20260811 07:38:58.648778 22020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:44841 every 8 connection(s)
I20260811 07:38:58.649492 21947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:38:58.652420 22021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:38:58.657302 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 21947
I20260811 07:38:58.657430 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:38:58.657784 18723 external_mini_cluster.cc:1428] Setting key 059961f1ec33eaefbc7695ceefb1c6db
I20260811 07:38:58.658581 22021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e: Bootstrap starting.
I20260811 07:38:58.661587 22021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e: Neither blocks nor log segments found. Creating new log.
I20260811 07:38:58.662729 22021 log.cc:824] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:38:58.665201 22021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e: No bootstrap required, opened a new log
Aug 11 07:38:58 dist-test-slave-zkpd krb5kdc[21929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:38:58.670073 22021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a627119ae94ce1ace8e12d3e3e665e" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44841 } }
I20260811 07:38:58.670426 22021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:38:58.670539 22021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a627119ae94ce1ace8e12d3e3e665e, State: Initialized, Role: FOLLOWER
I20260811 07:38:58.671175 22021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a627119ae94ce1ace8e12d3e3e665e" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44841 } }
I20260811 07:38:58.671394 22021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:38:58.671483 22021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:38:58.671591 22021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:38:58.672892 22021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a627119ae94ce1ace8e12d3e3e665e" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44841 } }
I20260811 07:38:58.673287 22021 leader_election.cc:302] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98a627119ae94ce1ace8e12d3e3e665e; no voters:
I20260811 07:38:58.673641 22021 leader_election.cc:288] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:38:58.673777 22026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:38:58.674064 22026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [term 1 LEADER]: Becoming Leader. State: Replica: 98a627119ae94ce1ace8e12d3e3e665e, State: Running, Role: LEADER
I20260811 07:38:58.674397 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:58.659121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.18.72.254:44841)
0811 07:38:58.659564 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:58.659584 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:58.659636 (+ 52us) server_negotiation.cc:407] Connection header received
0811 07:38:58.660522 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:58.660545 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:58.660959 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:58.661348 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:58.662143 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:58.663595 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:58.664660 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:58.665411 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:58.665602 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:58.668425 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:58.668433 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:58.668462 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:38:58.668503 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:58.670643 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:58.671317 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:58.671325 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:58.671362 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:58.671584 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:38:58.671936 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:38:58.671942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:38:58.671947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:38:58.672499 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:58.672725 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:58.672963 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 07:38:58.673297 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":117,"threads_started":1}
I20260811 07:38:58.674437 22026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a627119ae94ce1ace8e12d3e3e665e" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44841 } }
I20260811 07:38:58.675012 22021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:38:58.676836 22027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98a627119ae94ce1ace8e12d3e3e665e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a627119ae94ce1ace8e12d3e3e665e" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44841 } } }
I20260811 07:38:58.676987 22027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:58.677346 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:38:58.677342 22028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98a627119ae94ce1ace8e12d3e3e665e. Latest consensus state: current_term: 1 leader_uuid: "98a627119ae94ce1ace8e12d3e3e665e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98a627119ae94ce1ace8e12d3e3e665e" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44841 } } }
I20260811 07:38:58.677731 22028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e [sys.catalog]: This master's current role is: LEADER
I20260811 07:38:58.677979 22041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:38:58.681064 22041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:38:58.686169 22041 catalog_manager.cc:1347] Generated new cluster ID: 79160e76fa7e46498314f2e11213b9ea
I20260811 07:38:58.686239 22041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:38:58.696300 22041 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:38:58.697590 22041 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:38:58.703804 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:38:58.698277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.18.72.254:44841)
0811 07:38:58.698401 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 07:38:58.698406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:38:58.698423 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:38:58.698546 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:38:58.698549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:38:58.698598 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:38:58.698687 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:38:58.699272 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:58.700126 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:58.701283 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:38:58.701676 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:38:58.701776 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:38:58.702764 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:38:58.702770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:38:58.702775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:38:58.703350 (+ 575us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:38:58.703385 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:38:58.703406 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:38:58.703494 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:38:58.703602 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 07:38:58.703663 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:38:58.707834 22041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98a627119ae94ce1ace8e12d3e3e665e: Generated new TSK 0
I20260811 07:38:58.708685 22041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:38:58.717576 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 21947
2026-08-11T07:38:58Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1479 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:38:58 dist-test-slave-zkpd krb5kdc[22057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:38:58 dist-test-slave-zkpd krb5kdc[22057](info): set up 2 sockets
Aug 11 07:38:58 dist-test-slave-zkpd krb5kdc[22057](info): commencing operation
krb5kdc: starting...
W20260811 07:38:59.810010 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:38:59 dist-test-slave-zkpd krb5kdc[22057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:38:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:38:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:38:59.935202 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41243
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:41243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:00.050487 22070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:00.050787 22070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:00.050861 22070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:00.054865 22070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:00.054953 22070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:00.054993 22070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:00.055055 22070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:00.055095 22070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:00.060288 22070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:41243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:00.061393 22070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:00.062422 22070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:00.071139 22075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:00.071211 22078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:00.071206 22076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:00.071419 22070 server_base.cc:1034] running on GCE node
I20260811 07:39:00.072135 22070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:00.073290 22070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:00.074467 22070 hybrid_clock.cc:648] HybridClock initialized: now 1786433940074451 us; error 26 us; skew 500 ppm
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:00.077247 22070 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:39:00.078176 22070 webserver.cc:460] Webserver started at http://127.18.72.254:33981/ using document root <none> and password file <none>
I20260811 07:39:00.078745 22070 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:00.078861 22070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:00.079097 22070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:00.081239 22070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "73bea7b5673a4e03b2e7c7a6ca94f398"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "f08af1b4959ec8089809391cd80240c3"
server_key_iv: "b7867f6a85c3c2512a3fe6b7c9ce4b0e"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.081853 22070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "73bea7b5673a4e03b2e7c7a6ca94f398"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "f08af1b4959ec8089809391cd80240c3"
server_key_iv: "b7867f6a85c3c2512a3fe6b7c9ce4b0e"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.085587 22070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:39:00.088138 22086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.089344 22070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:00.089483 22070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "73bea7b5673a4e03b2e7c7a6ca94f398"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "f08af1b4959ec8089809391cd80240c3"
server_key_iv: "b7867f6a85c3c2512a3fe6b7c9ce4b0e"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.089594 22070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:00.100630 22070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:00.103847 22070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:00.104113 22070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:00.111441 22070 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:41243
I20260811 07:39:00.111727 22144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:41243 every 8 connection(s)
I20260811 07:39:00.112538 22070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:00.115495 22145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:00.121284 22145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398: Bootstrap starting.
I20260811 07:39:00.122155 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 22070
I20260811 07:39:00.122306 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:00.122583 18723 external_mini_cluster.cc:1428] Setting key daa0db9ebfb4e222b2231336f2286ae9
I20260811 07:39:00.124282 22145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:00.125218 22145 log.cc:824] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:00.127530 22145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398: No bootstrap required, opened a new log
I20260811 07:39:00.131917 22145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bea7b5673a4e03b2e7c7a6ca94f398" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41243 } }
I20260811 07:39:00.132221 22145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:00.132282 22145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73bea7b5673a4e03b2e7c7a6ca94f398, State: Initialized, Role: FOLLOWER
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:00.132833 22145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bea7b5673a4e03b2e7c7a6ca94f398" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41243 } }
I20260811 07:39:00.133024 22145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:00.133131 22145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:00.133257 22145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:00.134492 22145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bea7b5673a4e03b2e7c7a6ca94f398" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41243 } }
I20260811 07:39:00.134864 22145 leader_election.cc:302] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73bea7b5673a4e03b2e7c7a6ca94f398; no voters:
I20260811 07:39:00.135210 22145 leader_election.cc:288] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:00.135301 22150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:00.135553 22150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [term 1 LEADER]: Becoming Leader. State: Replica: 73bea7b5673a4e03b2e7c7a6ca94f398, State: Running, Role: LEADER
I20260811 07:39:00.135872 22150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bea7b5673a4e03b2e7c7a6ca94f398" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41243 } }
I20260811 07:39:00.136698 22145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:00.137751 22151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73bea7b5673a4e03b2e7c7a6ca94f398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bea7b5673a4e03b2e7c7a6ca94f398" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41243 } } }
I20260811 07:39:00.137751 22152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73bea7b5673a4e03b2e7c7a6ca94f398. Latest consensus state: current_term: 1 leader_uuid: "73bea7b5673a4e03b2e7c7a6ca94f398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73bea7b5673a4e03b2e7c7a6ca94f398" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41243 } } }
I20260811 07:39:00.137916 22151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:00.137915 22152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:00.138303 22159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:00.138942 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:00.123827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.18.72.254:41243)
0811 07:39:00.124405 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:00.124427 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:00.124476 (+ 49us) server_negotiation.cc:407] Connection header received
0811 07:39:00.125403 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:00.125427 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:00.125834 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:00.126207 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:00.126903 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.128407 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.129319 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.130028 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.130227 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:00.133072 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:00.133094 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:00.133115 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:00.133148 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:00.135111 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.135687 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.135693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.135700 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.135836 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.136254 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.136258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.136260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.136785 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:00.136993 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:00.137493 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 07:39:00.137858 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":144,"threads_started":1}
I20260811 07:39:00.141638 22159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:00.148029 22159 catalog_manager.cc:1347] Generated new cluster ID: e69e349a1b52494987790ca6c41ba7b4
I20260811 07:39:00.148108 22159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:00.166782 22159 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:00.168205 22159 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:00.175773 22159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73bea7b5673a4e03b2e7c7a6ca94f398: Generated new TSK 0
I20260811 07:39:00.176556 22159 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:39:00.206617 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41243
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:00.324656 22171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:00.324965 22171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:00.325040 22171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:00.329085 22171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:00.329169 22171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:00.329264 22171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:00.334594 22171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:00.335695 22171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:00.336843 22171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:00.346575 22176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:00.346704 22171 server_base.cc:1034] running on GCE node
W20260811 07:39:00.346591 22177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:00.346887 22179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:00.347326 22171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:00.348109 22171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:00.349274 22171 hybrid_clock.cc:648] HybridClock initialized: now 1786433940349256 us; error 29 us; skew 500 ppm
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:00.352155 22171 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:39:00.352995 22171 webserver.cc:460] Webserver started at http://127.18.72.193:44901/ using document root <none> and password file <none>
I20260811 07:39:00.353569 22171 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:00.353631 22171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:00.353820 22171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:00.355805 22171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "e948d679e59244ad82ae9eccaaa6ad74"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "8567ff990b559e68c2eeefaa1f8b2038"
server_key_iv: "75115eb59dc3493cc515125790168247"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.356437 22171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "e948d679e59244ad82ae9eccaaa6ad74"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "8567ff990b559e68c2eeefaa1f8b2038"
server_key_iv: "75115eb59dc3493cc515125790168247"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.360164 22171 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:00.362550 22186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.363771 22171 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:00.363960 22171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "e948d679e59244ad82ae9eccaaa6ad74"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "8567ff990b559e68c2eeefaa1f8b2038"
server_key_iv: "75115eb59dc3493cc515125790168247"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.364071 22171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:00.382741 22171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:00.385849 22171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:00.386067 22171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:00.386701 22171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:00.387601 22171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:00.387678 22171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.387753 22171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:00.387797 22171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.397644 22171 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:41441
I20260811 07:39:00.397671 22311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:41441 every 8 connection(s)
I20260811 07:39:00.398639 22171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:00.404162 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 22171
I20260811 07:39:00.404309 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:00.404616 18723 external_mini_cluster.cc:1428] Setting key af4dd5b3217fb442e8c4c58035a10a12
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:00.413107 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:00.400496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:59979 (local address 127.18.72.254:41243)
0811 07:39:00.400640 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:00.400644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:00.401528 (+ 884us) server_negotiation.cc:407] Connection header received
0811 07:39:00.402923 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:00.402926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:00.402982 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:00.403095 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:00.404387 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.405003 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.406404 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.406630 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.406742 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:00.409253 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:00.409261 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:00.409263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:00.409288 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:00.410712 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.411394 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.411397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.411398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.411471 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.411868 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.411870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.411871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.412102 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:00.412182 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:00.412873 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 07:39:00.412974 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:39:00.414422 22314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:00.400914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41243 (local address 127.18.72.193:59979)
0811 07:39:00.401380 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:00.401425 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:00.402705 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:00.403277 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:00.403285 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:00.403650 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:00.404201 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:00.404215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.405139 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.405142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:00.406243 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:00.406253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.406787 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.406791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:00.406972 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:00.407580 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:00.407597 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:00.409087 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:00.410846 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:00.410856 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:00.410876 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:00.411273 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:00.411572 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:00.411575 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:00.411577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:00.411770 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:00.412221 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:00.412231 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:00.412597 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 07:39:00.412840 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 07:39:00.413255 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":105,"threads_started":1}
I20260811 07:39:00.415768 22312 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41243
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
I20260811 07:39:00.416057 22312 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:00.416559 22312 heartbeater.cc:511] Master 127.18.72.254:41243 requested a full tablet report, sending...
I20260811 07:39:00.418124 22106 ts_manager.cc:194] Registered new tserver with Master: e948d679e59244ad82ae9eccaaa6ad74 (127.18.72.193:41441)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
I20260811 07:39:00.419476 22106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:59979
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:39:00.435002 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41243
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:00.550168 22317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:00.550472 22317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:00.550571 22317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:00.554672 22317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:00.554757 22317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:00.554848 22317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:00.560214 22317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:00.561314 22317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:00.562366 22317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:00.571995 22327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:00.572000 22325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:00.572116 22317 server_base.cc:1034] running on GCE node
W20260811 07:39:00.572000 22324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:00.572691 22317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:00.573335 22317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:00.574497 22317 hybrid_clock.cc:648] HybridClock initialized: now 1786433940574481 us; error 26 us; skew 500 ppm
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:00.577399 22317 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:39:00.578292 22317 webserver.cc:460] Webserver started at http://127.18.72.194:37277/ using document root <none> and password file <none>
I20260811 07:39:00.578840 22317 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:00.578902 22317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:00.579078 22317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:00.581213 22317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "5ba671fe9a9f4c828c982b6b0bc8961b"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "2c4743ae17d450addc80526e2f4f881a"
server_key_iv: "831f0498040dc54d11cb4e38f1609370"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.581792 22317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "5ba671fe9a9f4c828c982b6b0bc8961b"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "2c4743ae17d450addc80526e2f4f881a"
server_key_iv: "831f0498040dc54d11cb4e38f1609370"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.585621 22317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 07:39:00.588017 22334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.589241 22317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:39:00.589399 22317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "5ba671fe9a9f4c828c982b6b0bc8961b"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "2c4743ae17d450addc80526e2f4f881a"
server_key_iv: "831f0498040dc54d11cb4e38f1609370"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.589511 22317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:00.620308 22317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:00.624048 22317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:00.624269 22317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:00.624922 22317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:00.625833 22317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:00.625903 22317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.625978 22317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:00.626022 22317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.635354 22317 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:38531
I20260811 07:39:00.635381 22462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:38531 every 8 connection(s)
I20260811 07:39:00.636296 22317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:00.642761 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 22317
I20260811 07:39:00.642896 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:00.643255 18723 external_mini_cluster.cc:1428] Setting key 066d69843dfe7a87f6aa78440565a230
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:00.652879 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:00.638407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:36167 (local address 127.18.72.254:41243)
0811 07:39:00.638541 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:00.638545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:00.639544 (+ 999us) server_negotiation.cc:407] Connection header received
0811 07:39:00.641068 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:00.641071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:00.641118 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:00.641194 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:00.642444 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.643202 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.644814 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.645070 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.645154 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:00.648497 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:00.648501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:00.648504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:00.648537 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:00.650250 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.650905 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.650912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.650917 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.651022 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.651459 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.651465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.651469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.651762 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:00.651886 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:00.652477 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 07:39:00.652631 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:39:00.653510 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:00.638874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41243 (local address 127.18.72.194:36167)
0811 07:39:00.639356 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:00.639400 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:00.640870 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:00.641372 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:00.641379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:00.641809 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:00.642302 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:00.642315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.643385 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.643391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:00.644683 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:00.644697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.645234 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.645239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:00.645466 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:00.646108 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:00.646133 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:00.648198 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:00.650375 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:00.650380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:00.650394 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:00.650650 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:00.651150 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:00.651153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:00.651154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:00.651297 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:00.651930 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:00.651944 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:00.652191 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 07:39:00.652358 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 07:39:00.652627 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":116,"threads_started":1}
I20260811 07:39:00.654961 22463 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41243
I20260811 07:39:00.655222 22463 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:39:00.655766 22463 heartbeater.cc:511] Master 127.18.72.254:41243 requested a full tablet report, sending...
I20260811 07:39:00.656967 22106 ts_manager.cc:194] Registered new tserver with Master: 5ba671fe9a9f4c828c982b6b0bc8961b (127.18.72.194:38531)
I20260811 07:39:00.657563 22106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:36167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:39:00.674463 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41243
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:00.793309 22470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:00.793653 22470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:00.793766 22470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:00.797919 22470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:00.797978 22470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:00.798036 22470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:00.803448 22470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:00.804608 22470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:00.805676 22470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:00.815439 22475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:00.815531 22470 server_base.cc:1034] running on GCE node
W20260811 07:39:00.815475 22478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:00.815475 22476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:00.816141 22470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:00.816740 22470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:00.817893 22470 hybrid_clock.cc:648] HybridClock initialized: now 1786433940817861 us; error 45 us; skew 500 ppm
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:00.820652 22470 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:39:00.821589 22470 webserver.cc:460] Webserver started at http://127.18.72.195:40753/ using document root <none> and password file <none>
I20260811 07:39:00.822142 22470 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:00.822203 22470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:00.822372 22470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:00.824576 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "c21f321818074d449db1015d75e8ed43"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "97092a9f10fab2d49ac99608f8deba91"
server_key_iv: "73048a0fd9208a8c6ccddb7e09b2b3aa"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.825137 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "c21f321818074d449db1015d75e8ed43"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "97092a9f10fab2d49ac99608f8deba91"
server_key_iv: "73048a0fd9208a8c6ccddb7e09b2b3aa"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.828948 22470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 07:39:00.831435 22487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.832746 22470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:00.832890 22470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "c21f321818074d449db1015d75e8ed43"
format_stamp: "Formatted at 2026-08-11 07:39:00 on dist-test-slave-zkpd"
server_key: "97092a9f10fab2d49ac99608f8deba91"
server_key_iv: "73048a0fd9208a8c6ccddb7e09b2b3aa"
server_key_version: "encryptionkey@0"
I20260811 07:39:00.832998 22470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:00.847244 22470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:00.850739 22470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:00.850955 22470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:00.851562 22470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:00.852506 22470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:00.852562 22470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.852648 22470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:00.852687 22470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:00.861759 22470 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:37749
I20260811 07:39:00.861769 22612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:37749 every 8 connection(s)
I20260811 07:39:00.862720 22470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:00.872083 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 22470
I20260811 07:39:00.872219 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:00.872601 18723 external_mini_cluster.cc:1428] Setting key bd2300b53ad098feb0e3bc22d2f490bb
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:00.877691 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:00.864733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:38525 (local address 127.18.72.254:41243)
0811 07:39:00.864884 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:00.864888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:00.865793 (+ 905us) server_negotiation.cc:407] Connection header received
0811 07:39:00.867209 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:00.867213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:00.867271 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:00.867385 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:00.868883 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.869412 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.870570 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.870810 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.870952 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:00.873902 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:00.873905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:00.873908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:00.873931 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:00.875509 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.876113 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.876117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.876119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.876203 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.876572 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.876575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.876577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.876764 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:00.876880 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:00.877403 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 07:39:00.877522 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:39:00.878484 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:00.865144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41243 (local address 127.18.72.195:38525)
0811 07:39:00.865635 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:00.865680 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:00.866986 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:00.867581 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:00.867589 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:00.868094 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:00.868692 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:00.868713 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.869549 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.869553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:00.870433 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:00.870443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.870996 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.871000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:00.871172 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:00.871778 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:00.871802 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:00.873717 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:00.875653 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:00.875659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:00.875675 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:00.876006 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:00.876312 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:00.876315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:00.876317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:00.876464 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:00.876890 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:00.876899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:00.877148 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 07:39:00.877342 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 07:39:00.877667 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":122,"threads_started":1}
I20260811 07:39:00.879644 22613 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41243
I20260811 07:39:00.879940 22613 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:00.880448 22613 heartbeater.cc:511] Master 127.18.72.254:41243 requested a full tablet report, sending...
I20260811 07:39:00.881475 22106 ts_manager.cc:194] Registered new tserver with Master: c21f321818074d449db1015d75e8ed43 (127.18.72.195:37749)
I20260811 07:39:00.882045 22106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:38525
I20260811 07:39:00.886474 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 07:39:00 dist-test-slave-zkpd krb5kdc[22057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433940, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:00.914170 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:00.905550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.18.72.254:41243)
0811 07:39:00.905741 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:00.905746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:00.905904 (+ 158us) server_negotiation.cc:407] Connection header received
0811 07:39:00.906090 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:00.906093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:00.906138 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:00.906231 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:00.906985 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.907512 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.908454 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:00.908674 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:00.908787 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:00.911132 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:00.911135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:00.911137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:00.911161 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:00.912433 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.912851 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.912857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.912859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.912948 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:00.913229 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:00.913233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:00.913235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:00.913467 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:00.913612 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:00.913861 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 07:39:00.913999 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 07:39:00.914587 22106 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56120
I20260811 07:39:00.915478 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 22171
I20260811 07:39:00.925742 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 22317
I20260811 07:39:00.933642 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 22470
I20260811 07:39:00.941846 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 22070
2026-08-11T07:39:00Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2222 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:01 dist-test-slave-zkpd krb5kdc[22626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:01 dist-test-slave-zkpd krb5kdc[22626](info): set up 2 sockets
Aug 11 07:39:01 dist-test-slave-zkpd krb5kdc[22626](info): commencing operation
krb5kdc: starting...
W20260811 07:39:02.046070 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:02 dist-test-slave-zkpd krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:39:02.174829 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:34961
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:34961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:02.289157 22640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:02.289506 22640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:02.289593 22640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:02.293642 22640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:02.293779 22640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:02.293817 22640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:02.293854 22640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:02.293886 22640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:02.299289 22640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:34961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:02.300508 22640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:02.301595 22640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:02.310624 22646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:02.310655 22648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:02.310609 22645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:02.310957 22640 server_base.cc:1034] running on GCE node
I20260811 07:39:02.311625 22640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:02.312840 22640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:02.314042 22640 hybrid_clock.cc:648] HybridClock initialized: now 1786433942314025 us; error 34 us; skew 500 ppm
Aug 11 07:39:02 dist-test-slave-zkpd krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:02.317126 22640 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:39:02.318141 22640 webserver.cc:460] Webserver started at http://127.18.72.254:33097/ using document root <none> and password file <none>
I20260811 07:39:02.318773 22640 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:02.318874 22640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:02.319134 22640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:02.321367 22640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "ea492971fe0044bbbda5400bd7833227"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "4f3cbb2eab197627b89d6ded29dd9ea3"
server_key_iv: "9fde1c6f008f1b48ae22a4077ed582da"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.321959 22640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "ea492971fe0044bbbda5400bd7833227"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "4f3cbb2eab197627b89d6ded29dd9ea3"
server_key_iv: "9fde1c6f008f1b48ae22a4077ed582da"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.326122 22640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 07:39:02.328720 22655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:02.330006 22640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:39:02.330173 22640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "ea492971fe0044bbbda5400bd7833227"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "4f3cbb2eab197627b89d6ded29dd9ea3"
server_key_iv: "9fde1c6f008f1b48ae22a4077ed582da"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.330317 22640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:02.349068 22640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:02.352015 22640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:02.352250 22640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:02.359616 22640 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:34961
I20260811 07:39:02.359671 22713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:34961 every 8 connection(s)
I20260811 07:39:02.360646 22640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:02.361007 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 22640
I20260811 07:39:02.361153 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:02.361464 18723 external_mini_cluster.cc:1428] Setting key 6516910481335c0d92b747c703f7b489
I20260811 07:39:02.363838 22714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:02.369761 22714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227: Bootstrap starting.
Aug 11 07:39:02 dist-test-slave-zkpd krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:02.372354 22714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:02.373158 22714 log.cc:824] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:02.375538 22714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227: No bootstrap required, opened a new log
I20260811 07:39:02.376068 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:02.362910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.18.72.254:34961)
0811 07:39:02.363357 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:02.363385 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:02.363421 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:39:02.364148 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:02.364167 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:02.364515 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:02.364871 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:02.365684 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.366666 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.367525 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.368076 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.368250 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:02.370840 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:02.370845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:02.370858 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:02.370885 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:02.372687 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:02.373215 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:02.373221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:02.373226 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:02.373391 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:02.373722 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:02.373724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:02.373725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:02.374263 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:02.374455 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:02.374746 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 07:39:02.375065 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":146,"threads_started":1}
I20260811 07:39:02.379680 22714 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea492971fe0044bbbda5400bd7833227" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34961 } }
I20260811 07:39:02.379961 22714 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:02.380021 22714 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea492971fe0044bbbda5400bd7833227, State: Initialized, Role: FOLLOWER
I20260811 07:39:02.380466 22714 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea492971fe0044bbbda5400bd7833227" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34961 } }
I20260811 07:39:02.380590 22714 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:02.380636 22714 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:02.380708 22714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:02.381594 22714 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea492971fe0044bbbda5400bd7833227" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34961 } }
I20260811 07:39:02.381989 22714 leader_election.cc:302] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea492971fe0044bbbda5400bd7833227; no voters:
I20260811 07:39:02.382393 22714 leader_election.cc:288] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:02.382496 22719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:02.382732 22719 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [term 1 LEADER]: Becoming Leader. State: Replica: ea492971fe0044bbbda5400bd7833227, State: Running, Role: LEADER
I20260811 07:39:02.383086 22719 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea492971fe0044bbbda5400bd7833227" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34961 } }
I20260811 07:39:02.383697 22714 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:02.384805 22720 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea492971fe0044bbbda5400bd7833227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea492971fe0044bbbda5400bd7833227" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34961 } } }
I20260811 07:39:02.384938 22720 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:02.385337 22730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:02.385299 22721 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea492971fe0044bbbda5400bd7833227. Latest consensus state: current_term: 1 leader_uuid: "ea492971fe0044bbbda5400bd7833227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea492971fe0044bbbda5400bd7833227" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34961 } } }
I20260811 07:39:02.385525 22721 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:02.388275 22730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:02.393942 22730 catalog_manager.cc:1347] Generated new cluster ID: 66af5eaf055e46a19e0c5e23aa2e4512
I20260811 07:39:02.394011 22730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:02.408751 22730 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:02.410023 22730 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:02.420396 22730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea492971fe0044bbbda5400bd7833227: Generated new TSK 0
I20260811 07:39:02.421212 22730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:39:02.454761 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34961
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:02.571142 22745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:02.571395 22745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:02.571439 22745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:02.575474 22745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:02.575536 22745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:02.575632 22745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:02.581130 22745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:02.582237 22745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:02.583343 22745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:02.593196 22754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:02.593158 22752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:02.593223 22750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:02.593314 22745 server_base.cc:1034] running on GCE node
I20260811 07:39:02.593801 22745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:02.594444 22745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:02.595633 22745 hybrid_clock.cc:648] HybridClock initialized: now 1786433942595598 us; error 46 us; skew 500 ppm
Aug 11 07:39:02 dist-test-slave-zkpd krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:02.598380 22745 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:39:02.599267 22745 webserver.cc:460] Webserver started at http://127.18.72.193:44107/ using document root <none> and password file <none>
I20260811 07:39:02.599846 22745 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:02.599964 22745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:02.600205 22745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:02.602349 22745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "a70e7b07266f472082167754b66028f3"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "ec4f92a5f73a2e52ff7c40e06c34ea2a"
server_key_iv: "ceb416666bbdee5c96e10dcdc4256e7c"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.602952 22745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "a70e7b07266f472082167754b66028f3"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "ec4f92a5f73a2e52ff7c40e06c34ea2a"
server_key_iv: "ceb416666bbdee5c96e10dcdc4256e7c"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.607120 22745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 07:39:02.609686 22761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:02.610888 22745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:02.611027 22745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "a70e7b07266f472082167754b66028f3"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "ec4f92a5f73a2e52ff7c40e06c34ea2a"
server_key_iv: "ceb416666bbdee5c96e10dcdc4256e7c"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.611137 22745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:02.627246 22745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:02.630529 22745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:02.630725 22745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:02.631310 22745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:02.632197 22745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:02.632266 22745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:02.632337 22745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:02.632390 22745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:02.641287 22745 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:45805
I20260811 07:39:02.641309 22883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:45805 every 8 connection(s)
I20260811 07:39:02.642180 22745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:39:02 dist-test-slave-zkpd krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:02.652148 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 22745
I20260811 07:39:02.652274 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:02.652577 18723 external_mini_cluster.cc:1428] Setting key c665b88fdd100478d5566aca461ec000
I20260811 07:39:02.656070 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:02.644100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:56477 (local address 127.18.72.254:34961)
0811 07:39:02.644286 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:02.644291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:02.645319 (+ 1028us) server_negotiation.cc:407] Connection header received
0811 07:39:02.646691 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:02.646694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:02.646742 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:02.646815 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:02.648035 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.648574 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.649668 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.649900 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.649977 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:02.652437 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:02.652440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:02.652442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:02.652465 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:02.653973 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:02.654506 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:02.654509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:02.654510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:02.654586 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:02.654936 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:02.654939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:02.654941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:02.655120 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:02.655233 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:02.655735 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 07:39:02.655874 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:39:02.656885 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:02.644615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:34961 (local address 127.18.72.193:56477)
0811 07:39:02.645170 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:02.645211 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:02.646499 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:02.646988 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:02.646994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:02.647369 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:02.647847 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:02.647860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.648684 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.648687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:02.649564 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:02.649573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.650030 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.650033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:02.650167 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:02.650718 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:02.650734 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:02.652223 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:02.654107 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:02.654113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:02.654129 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:02.654390 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:02.654689 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:02.654691 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:02.654693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:02.654835 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:02.655243 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:02.655252 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:02.655498 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 07:39:02.655677 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 07:39:02.656021 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":157,"threads_started":1}
I20260811 07:39:02.658123 22886 heartbeater.cc:347] Connected to a master server at 127.18.72.254:34961
I20260811 07:39:02.658384 22886 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:02.658864 22886 heartbeater.cc:511] Master 127.18.72.254:34961 requested a full tablet report, sending...
I20260811 07:39:02.660709 22674 ts_manager.cc:194] Registered new tserver with Master: a70e7b07266f472082167754b66028f3 (127.18.72.193:45805)
I20260811 07:39:02.662382 22674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:56477
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:39:02.682847 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34961
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:02.797935 22892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:02.798228 22892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:02.798308 22892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:02.802389 22892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:02.802474 22892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:02.802565 22892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:02.807796 22892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:02.808910 22892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:02.810021 22892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:02.820053 22900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:02.820154 22892 server_base.cc:1034] running on GCE node
W20260811 07:39:02.820124 22897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:02.820053 22898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:02.820789 22892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:02.821457 22892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:02.822620 22892 hybrid_clock.cc:648] HybridClock initialized: now 1786433942822599 us; error 29 us; skew 500 ppm
Aug 11 07:39:02 dist-test-slave-zkpd krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:02.825487 22892 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:39:02.826339 22892 webserver.cc:460] Webserver started at http://127.18.72.194:44585/ using document root <none> and password file <none>
I20260811 07:39:02.826891 22892 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:02.826951 22892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:02.827121 22892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:02.829238 22892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "2f49ffab896941c7851049752cb8ff2a"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "f3378965df0e7c7c5d5e800b7b6f92c1"
server_key_iv: "b101aafced83246a5a7ccd2c57f7a57e"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.829761 22892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "2f49ffab896941c7851049752cb8ff2a"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "f3378965df0e7c7c5d5e800b7b6f92c1"
server_key_iv: "b101aafced83246a5a7ccd2c57f7a57e"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.833483 22892 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:39:02.835958 22907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:02.837198 22892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:02.837319 22892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "2f49ffab896941c7851049752cb8ff2a"
format_stamp: "Formatted at 2026-08-11 07:39:02 on dist-test-slave-zkpd"
server_key: "f3378965df0e7c7c5d5e800b7b6f92c1"
server_key_iv: "b101aafced83246a5a7ccd2c57f7a57e"
server_key_version: "encryptionkey@0"
I20260811 07:39:02.837431 22892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:02.855840 22892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:02.858810 22892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:02.859028 22892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:02.859647 22892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:02.860509 22892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:02.860556 22892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:02.860596 22892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:02.860656 22892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:02.869773 22892 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:38613
I20260811 07:39:02.869839 23026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:38613 every 8 connection(s)
I20260811 07:39:02.870673 22892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:02.879815 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 22892
I20260811 07:39:02.879961 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:02.880347 18723 external_mini_cluster.cc:1428] Setting key d91da34ff52456567774aa215145b8eb
Aug 11 07:39:02 dist-test-slave-zkpd krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433942, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:02.885393 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:02.872597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:33513 (local address 127.18.72.254:34961)
0811 07:39:02.872744 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:02.872750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:02.873599 (+ 849us) server_negotiation.cc:407] Connection header received
0811 07:39:02.875090 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:02.875094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:02.875158 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:02.875260 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:02.876499 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.877023 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.878287 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.878497 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.878575 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:02.881362 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:02.881366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:02.881368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:02.881392 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:02.882873 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:02.883494 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:02.883497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:02.883498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:02.883573 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:02.884065 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:02.884071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:02.884075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:02.884352 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:02.884460 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:02.885029 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 07:39:02.885163 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:39:02.886234 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:02.872963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:34961 (local address 127.18.72.194:33513)
0811 07:39:02.873437 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:02.873485 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:02.874867 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:02.875442 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:02.875449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:02.875819 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:02.876355 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:02.876370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.877230 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.877233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:02.878174 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:02.878184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:02.878629 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:02.878631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:02.878803 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:02.879422 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:02.879438 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:02.881200 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:02.882995 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:02.883003 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:02.883022 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:02.883397 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:02.883668 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:02.883674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:02.883679 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:02.883940 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:02.884471 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:02.884481 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:02.884730 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 07:39:02.884947 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 07:39:02.885202 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":108,"threads_started":1}
I20260811 07:39:02.887562 23027 heartbeater.cc:347] Connected to a master server at 127.18.72.254:34961
I20260811 07:39:02.887836 23027 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:02.888499 23027 heartbeater.cc:511] Master 127.18.72.254:34961 requested a full tablet report, sending...
I20260811 07:39:02.889706 22674 ts_manager.cc:194] Registered new tserver with Master: 2f49ffab896941c7851049752cb8ff2a (127.18.72.194:38613)
I20260811 07:39:02.890532 22674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:33513
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:39:02.911047 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34961
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:03.026777 23032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:03.027097 23032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:03.027176 23032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:03.031311 23032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:03.031400 23032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:03.031492 23032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:03.036988 23032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:03.038092 23032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:03.039156 23032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:03.049180 23039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:03.049149 23041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:03.049963 23038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:03.050262 23032 server_base.cc:1034] running on GCE node
I20260811 07:39:03.050705 23032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:03.051365 23032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:03.052551 23032 hybrid_clock.cc:648] HybridClock initialized: now 1786433943052536 us; error 38 us; skew 500 ppm
Aug 11 07:39:03 dist-test-slave-zkpd krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433943, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:03.055435 23032 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:39:03.056354 23032 webserver.cc:460] Webserver started at http://127.18.72.195:38977/ using document root <none> and password file <none>
I20260811 07:39:03.056929 23032 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:03.057022 23032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:03.057257 23032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:03.059353 23032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "e2bbdc33c7c8445e9156131d47442094"
format_stamp: "Formatted at 2026-08-11 07:39:03 on dist-test-slave-zkpd"
server_key: "5f41c4815af28a8561de85500b7d44fb"
server_key_iv: "15fa98201410de07fc2809f19a5019a8"
server_key_version: "encryptionkey@0"
I20260811 07:39:03.059958 23032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "e2bbdc33c7c8445e9156131d47442094"
format_stamp: "Formatted at 2026-08-11 07:39:03 on dist-test-slave-zkpd"
server_key: "5f41c4815af28a8561de85500b7d44fb"
server_key_iv: "15fa98201410de07fc2809f19a5019a8"
server_key_version: "encryptionkey@0"
I20260811 07:39:03.063673 23032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:03.066087 23048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:03.067250 23032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:03.067402 23032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "e2bbdc33c7c8445e9156131d47442094"
format_stamp: "Formatted at 2026-08-11 07:39:03 on dist-test-slave-zkpd"
server_key: "5f41c4815af28a8561de85500b7d44fb"
server_key_iv: "15fa98201410de07fc2809f19a5019a8"
server_key_version: "encryptionkey@0"
I20260811 07:39:03.067512 23032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:03.087805 23032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:03.091352 23032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:03.091572 23032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:03.092257 23032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:03.093117 23032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:03.093197 23032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:03.093274 23032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:03.093326 23032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:03.102367 23032 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:43957
I20260811 07:39:03.102396 23171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:43957 every 8 connection(s)
I20260811 07:39:03.103349 23032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:03.108587 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 23032
I20260811 07:39:03.108745 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:03.109081 18723 external_mini_cluster.cc:1428] Setting key 756beeab70d8a0af4bf4af7a21576ed1
Aug 11 07:39:03 dist-test-slave-zkpd krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433943, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:03.117511 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:03.105209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:50069 (local address 127.18.72.254:34961)
0811 07:39:03.105344 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:03.105347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:03.106212 (+ 865us) server_negotiation.cc:407] Connection header received
0811 07:39:03.107572 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:03.107576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:03.107669 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:03.107828 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:03.109339 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:03.110093 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:03.111362 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:03.111597 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:03.111706 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:03.114072 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:03.114076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:03.114079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:03.114105 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:03.115466 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:03.116002 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:03.116006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:03.116007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:03.116095 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:03.116443 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:03.116446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:03.116448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:03.116627 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:03.116734 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:03.117223 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 07:39:03.117344 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:39:03.118278 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:03.105591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:34961 (local address 127.18.72.195:50069)
0811 07:39:03.106046 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:03.106091 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:03.107326 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:03.108141 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:03.108149 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:03.108550 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:03.109164 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:03.109186 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:03.110226 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:03.110230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:03.111239 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:03.111249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:03.111759 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:03.111763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:03.111963 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:03.112495 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:03.112511 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:03.113906 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:03.115583 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:03.115589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:03.115605 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:03.115855 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:03.116194 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:03.116197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:03.116199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:03.116335 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:03.116732 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:03.116739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:03.116979 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 07:39:03.117165 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 07:39:03.117490 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":107,"threads_started":1}
I20260811 07:39:03.119370 23172 heartbeater.cc:347] Connected to a master server at 127.18.72.254:34961
I20260811 07:39:03.119590 23172 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:03.120100 23172 heartbeater.cc:511] Master 127.18.72.254:34961 requested a full tablet report, sending...
I20260811 07:39:03.121080 22674 ts_manager.cc:194] Registered new tserver with Master: e2bbdc33c7c8445e9156131d47442094 (127.18.72.195:43957)
I20260811 07:39:03.121675 22674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:50069
I20260811 07:39:03.122941 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 07:39:03 dist-test-slave-zkpd krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:39:03 dist-test-slave-zkpd krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:39:03.149621 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:03.140384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48740 (local address 127.18.72.193:45805)
0811 07:39:03.140672 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:03.140693 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:03.140747 (+ 54us) server_negotiation.cc:407] Connection header received
0811 07:39:03.140817 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:03.140823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:03.141017 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:03.141138 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:03.141933 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:03.142716 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:03.143612 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:03.144049 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:03.144133 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:03.146401 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:03.146410 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:03.146424 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:03.146448 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:03.147999 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:03.148411 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:03.148418 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:03.148419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:03.148509 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:03.148747 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:03.148749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:03.148750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:03.148999 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:03.149174 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:03.149318 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 07:39:03.149462 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
Aug 11 07:39:03 dist-test-slave-zkpd krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:03.160476 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:03.152616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.18.72.254:34961)
0811 07:39:03.152792 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:03.152796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:03.152812 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:39:03.152936 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:03.152938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:03.152980 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:03.153053 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:03.153613 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:03.154191 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:03.154946 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:03.155160 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:03.155254 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:03.157529 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:03.157532 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:03.157534 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:03.157557 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:03.158850 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:03.159232 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:03.159235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:03.159237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:03.159321 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:03.159551 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:03.159553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:03.159554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:03.159825 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:03.160013 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:03.160164 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 07:39:03.160269 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 07:39:03.160933 22674 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35972
I20260811 07:39:03.161602 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 22745
I20260811 07:39:03.170764 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 22892
I20260811 07:39:03.177186 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 23032
I20260811 07:39:03.184347 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 22640
2026-08-11T07:39:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2242 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:03 dist-test-slave-zkpd krb5kdc[23185](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:03 dist-test-slave-zkpd krb5kdc[23185](info): set up 2 sockets
Aug 11 07:39:03 dist-test-slave-zkpd krb5kdc[23185](info): commencing operation
krb5kdc: starting...
W20260811 07:39:04.278112 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:04 dist-test-slave-zkpd krb5kdc[23185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:39:04.405187 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:46123
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:46123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:04.519371 23201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:04.519662 23201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:04.519707 23201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:04.523921 23201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:04.523981 23201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:04.524003 23201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:04.524019 23201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:04.524077 23201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:04.529325 23201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:46123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:46123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:04.530535 23201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:04.531617 23201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:04.540210 23206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:04.540289 23207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:04.540205 23209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:04.540565 23201 server_base.cc:1034] running on GCE node
I20260811 07:39:04.541124 23201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:04.542176 23201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:04.543358 23201 hybrid_clock.cc:648] HybridClock initialized: now 1786433944543313 us; error 25 us; skew 500 ppm
Aug 11 07:39:04 dist-test-slave-zkpd krb5kdc[23185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433944, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:04.551507 23201 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:39:04.552371 23201 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:04.552469 23201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:04.552708 23201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:04.554790 23201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "21983cf91c9b4407bbddef8ba5af0176"
format_stamp: "Formatted at 2026-08-11 07:39:04 on dist-test-slave-zkpd"
server_key: "5e0a403f181c79ff943acef8c8c3f820"
server_key_iv: "ed470dae387637fca18fc052f451cd46"
server_key_version: "encryptionkey@0"
I20260811 07:39:04.555389 23201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "21983cf91c9b4407bbddef8ba5af0176"
format_stamp: "Formatted at 2026-08-11 07:39:04 on dist-test-slave-zkpd"
server_key: "5e0a403f181c79ff943acef8c8c3f820"
server_key_iv: "ed470dae387637fca18fc052f451cd46"
server_key_version: "encryptionkey@0"
I20260811 07:39:04.559295 23201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:39:04.561801 23215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:04.563090 23201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:04.563234 23201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "21983cf91c9b4407bbddef8ba5af0176"
format_stamp: "Formatted at 2026-08-11 07:39:04 on dist-test-slave-zkpd"
server_key: "5e0a403f181c79ff943acef8c8c3f820"
server_key_iv: "ed470dae387637fca18fc052f451cd46"
server_key_version: "encryptionkey@0"
I20260811 07:39:04.563351 23201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:04.574234 23201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:04.577438 23201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:04.577646 23201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:04.584698 23201 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:46123
I20260811 07:39:04.584977 23277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:46123 every 8 connection(s)
I20260811 07:39:04.585422 23201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:04.588372 23278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:04.591552 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 23201
I20260811 07:39:04.591689 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:04.592036 18723 external_mini_cluster.cc:1428] Setting key 74206a15323653d5be10e4d2e2e9d20a
I20260811 07:39:04.594686 23278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176: Bootstrap starting.
I20260811 07:39:04.597383 23278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:04.598289 23278 log.cc:824] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:04.600719 23278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176: No bootstrap required, opened a new log
Aug 11 07:39:04 dist-test-slave-zkpd krb5kdc[23185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:04.604130 23278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21983cf91c9b4407bbddef8ba5af0176" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46123 } }
I20260811 07:39:04.604382 23278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:04.604476 23278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21983cf91c9b4407bbddef8ba5af0176, State: Initialized, Role: FOLLOWER
I20260811 07:39:04.604990 23278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21983cf91c9b4407bbddef8ba5af0176" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46123 } }
I20260811 07:39:04.605188 23278 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:04.605261 23278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:04.605376 23278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:04.606248 23278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21983cf91c9b4407bbddef8ba5af0176" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46123 } }
I20260811 07:39:04.606604 23278 leader_election.cc:302] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21983cf91c9b4407bbddef8ba5af0176; no voters:
I20260811 07:39:04.606942 23278 leader_election.cc:288] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:04.607017 23283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:04.607213 23283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [term 1 LEADER]: Becoming Leader. State: Replica: 21983cf91c9b4407bbddef8ba5af0176, State: Running, Role: LEADER
I20260811 07:39:04.607522 23283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21983cf91c9b4407bbddef8ba5af0176" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46123 } }
I20260811 07:39:04.607628 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:04.593442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.18.72.254:46123)
0811 07:39:04.593864 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:04.593889 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:04.593925 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:39:04.594574 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:04.594592 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:04.594976 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:04.595262 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:04.596222 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:04.597213 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:04.598410 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:04.599117 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:04.599244 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:04.601988 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:04.601993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:04.602006 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:04.602031 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:04.604339 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:04.604862 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:04.604868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:04.604874 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:04.605017 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:04.605336 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:04.605342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:04.605344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:04.605787 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:04.606004 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:04.606314 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 07:39:04.606634 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":129,"threads_started":1}
I20260811 07:39:04.608269 23278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:04.609972 23283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21983cf91c9b4407bbddef8ba5af0176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21983cf91c9b4407bbddef8ba5af0176" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46123 } } }
I20260811 07:39:04.610101 23283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:04.610287 23283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21983cf91c9b4407bbddef8ba5af0176. Latest consensus state: current_term: 1 leader_uuid: "21983cf91c9b4407bbddef8ba5af0176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21983cf91c9b4407bbddef8ba5af0176" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46123 } } }
I20260811 07:39:04.610374 23283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:04.610648 23298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:04.613962 23298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:04.619841 23298 catalog_manager.cc:1347] Generated new cluster ID: 07c2d4ed27a94345ac8d19167fd23c41
I20260811 07:39:04.619933 23298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:04.628667 23298 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:04.629864 23298 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:04.646423 23298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176: Generated new TSK 0
I20260811 07:39:04.647173 23298 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:39:04.684705 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:46123
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:04.822263 23305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:04.822516 23305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:04.822561 23305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:04.826521 23305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:04.826586 23305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:04.826645 23305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:04.832449 23305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:04.833518 23305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:04.834618 23305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:04.844229 23310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:04.844275 23311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:04.844408 23305 server_base.cc:1034] running on GCE node
W20260811 07:39:04.844231 23313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:04.844981 23305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:04.845659 23305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:04.846859 23305 hybrid_clock.cc:648] HybridClock initialized: now 1786433944846851 us; error 30 us; skew 500 ppm
Aug 11 07:39:04 dist-test-slave-zkpd krb5kdc[23185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433944, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:04.849946 23305 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:39:04.850735 23305 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:04.850844 23305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:04.851085 23305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:04.853227 23305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "7cb9f31fecf74f8e98248171a4cb5532"
format_stamp: "Formatted at 2026-08-11 07:39:04 on dist-test-slave-zkpd"
server_key: "bd8c7866b8520d0d75153d4095e1ba63"
server_key_iv: "3d3f512effa655f1822598c856267074"
server_key_version: "encryptionkey@0"
I20260811 07:39:04.853825 23305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "7cb9f31fecf74f8e98248171a4cb5532"
format_stamp: "Formatted at 2026-08-11 07:39:04 on dist-test-slave-zkpd"
server_key: "bd8c7866b8520d0d75153d4095e1ba63"
server_key_iv: "3d3f512effa655f1822598c856267074"
server_key_version: "encryptionkey@0"
I20260811 07:39:04.857599 23305 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:39:04.860057 23319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:04.861308 23305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:04.861441 23305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "7cb9f31fecf74f8e98248171a4cb5532"
format_stamp: "Formatted at 2026-08-11 07:39:04 on dist-test-slave-zkpd"
server_key: "bd8c7866b8520d0d75153d4095e1ba63"
server_key_iv: "3d3f512effa655f1822598c856267074"
server_key_version: "encryptionkey@0"
I20260811 07:39:04.861549 23305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:04.889001 23305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:04.892616 23305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:04.892825 23305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:04.893982 23305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:04.894946 23305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:04.894994 23305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:04.895035 23305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:04.895049 23305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:04.904330 23305 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:38393
I20260811 07:39:04.904357 23443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:38393 every 8 connection(s)
I20260811 07:39:04.904930 23305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:04.912927 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 23305
I20260811 07:39:04.913076 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:04.913446 18723 external_mini_cluster.cc:1428] Setting key 97a6524c927827275f3f176abfcb9049
Aug 11 07:39:04 dist-test-slave-zkpd krb5kdc[23185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433944, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:04.919919 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:04.906703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:60391 (local address 127.18.72.254:46123)
0811 07:39:04.906905 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:04.906910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:04.907726 (+ 816us) server_negotiation.cc:407] Connection header received
0811 07:39:04.909188 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:04.909191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:04.909242 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:04.909318 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:04.910524 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:04.911291 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:04.912476 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:04.912691 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:04.912775 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:04.915814 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:04.915821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:04.915826 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:04.915859 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:04.917530 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:04.918210 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:04.918213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:04.918214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:04.918298 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:04.918696 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:04.918698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:04.918699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:04.918898 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:04.919003 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:04.919519 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 07:39:04.919658 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 07:39:04.920645 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:04.907102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:46123 (local address 127.18.72.193:60391)
0811 07:39:04.907565 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:04.907606 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:04.908981 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:04.909493 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:04.909500 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:04.909891 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:04.910362 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:04.910377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:04.911398 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:04.911401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:04.912366 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:04.912376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:04.912875 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:04.912881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:04.913106 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:04.913834 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:04.913855 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:04.915641 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:04.917656 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:04.917664 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:04.917680 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:04.918098 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:04.918392 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:04.918395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:04.918397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:04.918595 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:04.919040 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:04.919046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:04.919276 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 07:39:04.919436 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 07:39:04.919679 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":107,"threads_started":1}
I20260811 07:39:04.922032 23444 heartbeater.cc:347] Connected to a master server at 127.18.72.254:46123
I20260811 07:39:04.922277 23444 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:04.922714 23444 heartbeater.cc:511] Master 127.18.72.254:46123 requested a full tablet report, sending...
I20260811 07:39:04.924255 23235 ts_manager.cc:194] Registered new tserver with Master: 7cb9f31fecf74f8e98248171a4cb5532 (127.18.72.193:38393)
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
I20260811 07:39:04.925441 23235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:60391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:39:04.944085 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:46123
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:05.061030 23450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:05.061316 23450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:05.061365 23450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:05.065618 23450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:05.065686 23450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:05.065798 23450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:05.071183 23450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:05.072314 23450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:05.073371 23450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:05.082988 23457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:05.082989 23459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:05.082988 23455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:05.083252 23450 server_base.cc:1034] running on GCE node
I20260811 07:39:05.083794 23450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:05.084475 23450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:05.085664 23450 hybrid_clock.cc:648] HybridClock initialized: now 1786433945085646 us; error 35 us; skew 500 ppm
Aug 11 07:39:05 dist-test-slave-zkpd krb5kdc[23185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:05.088574 23450 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:39:05.089295 23450 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:05.089360 23450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:05.089526 23450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:05.091629 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "d8849ef02bc9485a8114397bb76795c8"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "e637685b6e46246ba3d0a7d84fc539eb"
server_key_iv: "c9895f09cbb8f37e6a7e745c2955056b"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.092214 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "d8849ef02bc9485a8114397bb76795c8"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "e637685b6e46246ba3d0a7d84fc539eb"
server_key_iv: "c9895f09cbb8f37e6a7e745c2955056b"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.095863 23450 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:39:05.098282 23467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:05.099583 23450 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:05.099678 23450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "d8849ef02bc9485a8114397bb76795c8"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "e637685b6e46246ba3d0a7d84fc539eb"
server_key_iv: "c9895f09cbb8f37e6a7e745c2955056b"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.099759 23450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:05.156898 23450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:05.160346 23450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:05.160513 23450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:05.161089 23450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:05.161927 23450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:05.161975 23450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:05.162015 23450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:05.162041 23450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:05.171723 23450 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:39937
I20260811 07:39:05.171736 23594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:39937 every 8 connection(s)
I20260811 07:39:05.172415 23450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:05.181993 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 23450
I20260811 07:39:05.182121 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
Aug 11 07:39:05 dist-test-slave-zkpd krb5kdc[23185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:05.182492 18723 external_mini_cluster.cc:1428] Setting key cc1d4271446c0e4189fa8df265ef13c1
I20260811 07:39:05.187342 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.174309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:44527 (local address 127.18.72.254:46123)
0811 07:39:05.174461 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.174465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.175388 (+ 923us) server_negotiation.cc:407] Connection header received
0811 07:39:05.176808 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.176811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.176871 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.176961 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:05.178200 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.178787 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.179982 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.180211 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.180330 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.183166 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:05.183169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:05.183171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:05.183199 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:05.184819 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:05.185462 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:05.185464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:05.185465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:05.185541 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:05.185982 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:05.185985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:05.185986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:05.186286 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:05.186422 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.186965 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.187132 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:39:05.188158 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.174710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:46123 (local address 127.18.72.194:44527)
0811 07:39:05.175227 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.175270 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:05.176552 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:05.177162 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:05.177170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:05.177554 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:05.178049 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.178062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.178935 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.178938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.179818 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.179827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.180378 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.180382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.180562 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.181218 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:05.181235 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:05.182979 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:05.184949 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:05.184958 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:05.184980 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:05.185362 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:05.185629 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:05.185635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:05.185640 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:05.185855 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:05.186429 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:05.186436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:05.186689 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 07:39:05.186864 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 07:39:05.187148 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":129,"threads_started":1}
I20260811 07:39:05.189333 23595 heartbeater.cc:347] Connected to a master server at 127.18.72.254:46123
I20260811 07:39:05.189628 23595 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:05.190238 23595 heartbeater.cc:511] Master 127.18.72.254:46123 requested a full tablet report, sending...
I20260811 07:39:05.191511 23235 ts_manager.cc:194] Registered new tserver with Master: d8849ef02bc9485a8114397bb76795c8 (127.18.72.194:39937)
I20260811 07:39:05.192176 23235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:44527
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:39:05.214715 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:46123
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:37583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:05.334146 23600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:05.334450 23600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:05.334498 23600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:05.338655 23600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:05.338719 23600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:05.338786 23600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:05.344272 23600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:46123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:05.345455 23600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:05.346599 23600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:05.356419 23607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:05.356565 23600 server_base.cc:1034] running on GCE node
W20260811 07:39:05.356419 23611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:05.356765 23608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:05.357224 23600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:05.357816 23600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:05.358976 23600 hybrid_clock.cc:648] HybridClock initialized: now 1786433945358960 us; error 27 us; skew 500 ppm
Aug 11 07:39:05 dist-test-slave-zkpd krb5kdc[23185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:05.362015 23600 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:39:05.362792 23600 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:05.362861 23600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:05.363090 23600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:05.365336 23600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "fa49d29c2f164e92a4c41788ccaa18d9"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "12b165bf67d71ae9a9d7a609c9479ea1"
server_key_iv: "be7421553bc089efcd53425d325be56e"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.365960 23600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "fa49d29c2f164e92a4c41788ccaa18d9"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "12b165bf67d71ae9a9d7a609c9479ea1"
server_key_iv: "be7421553bc089efcd53425d325be56e"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.369884 23600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:39:05.372452 23617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:05.373669 23600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:05.373848 23600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "fa49d29c2f164e92a4c41788ccaa18d9"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "12b165bf67d71ae9a9d7a609c9479ea1"
server_key_iv: "be7421553bc089efcd53425d325be56e"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.373946 23600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:05.424938 23600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:05.428575 23600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:05.428810 23600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:05.429461 23600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:05.430364 23600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:05.430415 23600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:05.430482 23600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:05.430516 23600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:05.439819 23600 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:35307
I20260811 07:39:05.439847 23740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:35307 every 8 connection(s)
I20260811 07:39:05.440495 23600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:05.442962 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 23600
I20260811 07:39:05.443110 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:05.443496 18723 external_mini_cluster.cc:1428] Setting key 389b4f954dfd30c383fd8c23e36db48b
Aug 11 07:39:05 dist-test-slave-zkpd krb5kdc[23185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:05.454373 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.442341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:35691 (local address 127.18.72.254:46123)
0811 07:39:05.442489 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.442493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.443406 (+ 913us) server_negotiation.cc:407] Connection header received
0811 07:39:05.444997 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.445000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.445079 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.445158 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:05.446407 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.446947 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.448122 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.448348 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.448428 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.450882 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:05.450886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:05.450888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:05.450912 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:05.452284 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:05.452813 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:05.452816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:05.452817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:05.452893 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:05.453247 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:05.453252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:05.453253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:05.453440 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:05.453547 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.454057 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.454189 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 07:39:05.455196 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.442743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:46123 (local address 127.18.72.195:35691)
0811 07:39:05.443228 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.443285 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:05.444775 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:05.445311 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:05.445318 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:05.445783 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:05.446276 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.446290 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.447054 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.447057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.448015 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.448026 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.448473 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.448476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.448609 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.449265 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:05.449282 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:05.450713 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:05.452402 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:05.452408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:05.452423 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:05.452685 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:05.452988 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:05.452992 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:05.452994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:05.453147 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:05.453559 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:05.453567 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:05.453821 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 07:39:05.454014 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 07:39:05.454407 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":125,"threads_started":1}
I20260811 07:39:05.456401 23741 heartbeater.cc:347] Connected to a master server at 127.18.72.254:46123
I20260811 07:39:05.456653 23741 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:05.457144 23741 heartbeater.cc:511] Master 127.18.72.254:46123 requested a full tablet report, sending...
I20260811 07:39:05.458221 23235 ts_manager.cc:194] Registered new tserver with Master: fa49d29c2f164e92a4c41788ccaa18d9 (127.18.72.195:35307)
I20260811 07:39:05.459105 23235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:35691
I20260811 07:39:05.469349 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:39:05.478942 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.471288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.18.72.254:46123)
0811 07:39:05.471404 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.471407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.471647 (+ 240us) server_negotiation.cc:407] Connection header received
0811 07:39:05.471798 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.471800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.471838 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.471963 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:05.472652 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.473164 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.474016 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.474240 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.474319 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.475688 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:05.475691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:05.475693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:05.475741 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:05.477158 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:05.477568 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:05.477571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:05.477572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:05.477771 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:05.478066 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:05.478069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:05.478070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:05.478294 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:05.478405 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.478656 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.478802 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:39:05.481844 23235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:39:05.484833 23235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:05.499866 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.493516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:39937 (local address 127.0.0.1:48810)
0811 07:39:05.494241 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.494297 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:05.494447 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:05.495323 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:05.495330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:05.495352 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:05.495532 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.495539 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.496712 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.496717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.498012 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.498024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.499229 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.499233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.499341 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.499398 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 07:39:05.499561 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 07:39:05.499680 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":33,"spinlock_wait_cycles":4864,"thread_start_us":90,"threads_started":1}
I20260811 07:39:05.500090 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.493970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.18.72.195:35307)
0811 07:39:05.494788 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.494805 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.494829 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:39:05.494917 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.494926 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.495164 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.495686 (+ 522us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:05.495934 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.497269 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.498647 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.499398 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.499511 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.499650 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.499808 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.499935 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":89,"threads_started":1}
I20260811 07:39:05.499969 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.493563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:35307 (local address 127.0.0.1:48198)
0811 07:39:05.494291 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.494307 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:05.494448 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:05.495340 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:05.495345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:05.495366 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:05.495538 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.495545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.497447 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.497451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.498520 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.498529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.499530 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.499537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.499641 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.499664 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 07:39:05.499752 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 07:39:05.499809 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":107,"spinlock_wait_cycles":3840,"thread_start_us":139,"threads_started":1}
I20260811 07:39:05.500752 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.494375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:38393 (local address 127.0.0.1:44498)
0811 07:39:05.495199 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.495230 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:05.495327 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:05.495866 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:05.495870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:05.495909 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:05.495999 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.496006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.497487 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.497490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.498793 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.498808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.500135 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.500142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.500425 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.500452 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 07:39:05.500530 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:39:05.500614 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":63,"thread_start_us":113,"threads_started":1}
I20260811 07:39:05.500916 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.493649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.18.72.194:39937)
0811 07:39:05.494788 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.494803 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.494827 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:39:05.494914 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.494920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.495139 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.495334 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:05.495651 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.496584 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.498153 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.499087 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.499220 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.499396 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.500488 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.500598 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":195,"threads_started":1}
I20260811 07:39:05.501231 23759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.494833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.18.72.193:38393)
0811 07:39:05.495138 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.495144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.495427 (+ 283us) server_negotiation.cc:407] Connection header received
0811 07:39:05.495518 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.495524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.495718 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.495873 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:05.496118 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.497360 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.498950 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.499797 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.499927 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.500066 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.500923 (+ 857us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.501043 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":72,"threads_started":1}
I20260811 07:39:05.503707 23523 tablet_service.cc:1467] Processing CreateTablet for tablet c423a0c9555a492d90c24dea5097d8a9 (DEFAULT_TABLE table=test-table [id=8f3883cd9e834bdf9238118f0240781c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:05.503995 23372 tablet_service.cc:1467] Processing CreateTablet for tablet c423a0c9555a492d90c24dea5097d8a9 (DEFAULT_TABLE table=test-table [id=8f3883cd9e834bdf9238118f0240781c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:05.504941 23523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c423a0c9555a492d90c24dea5097d8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:05.505112 23372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c423a0c9555a492d90c24dea5097d8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:05.505793 23668 tablet_service.cc:1467] Processing CreateTablet for tablet c423a0c9555a492d90c24dea5097d8a9 (DEFAULT_TABLE table=test-table [id=8f3883cd9e834bdf9238118f0240781c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:05.506752 23668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c423a0c9555a492d90c24dea5097d8a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:05.511866 23760 tablet_bootstrap.cc:492] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532: Bootstrap starting.
I20260811 07:39:05.513487 23762 tablet_bootstrap.cc:492] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: Bootstrap starting.
I20260811 07:39:05.513485 23761 tablet_bootstrap.cc:492] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: Bootstrap starting.
I20260811 07:39:05.514598 23760 tablet_bootstrap.cc:654] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:05.515611 23762 tablet_bootstrap.cc:654] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:05.515640 23760 log.cc:824] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:05.515875 23761 tablet_bootstrap.cc:654] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:05.516950 23762 log.cc:824] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:05.517156 23761 log.cc:824] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:05.518232 23760 tablet_bootstrap.cc:492] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532: No bootstrap required, opened a new log
I20260811 07:39:05.518467 23760 ts_tablet_manager.cc:1397] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260811 07:39:05.518999 23761 tablet_bootstrap.cc:492] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: No bootstrap required, opened a new log
I20260811 07:39:05.519084 23762 tablet_bootstrap.cc:492] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: No bootstrap required, opened a new log
I20260811 07:39:05.519224 23761 ts_tablet_manager.cc:1397] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 07:39:05.519299 23762 ts_tablet_manager.cc:1397] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 07:39:05.522449 23761 raft_consensus.cc:348] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.522712 23761 raft_consensus.cc:374] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:05.522797 23761 raft_consensus.cc:729] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8849ef02bc9485a8114397bb76795c8, State: Initialized, Role: FOLLOWER
I20260811 07:39:05.523286 23761 consensus_queue.cc:260] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.523518 23762 raft_consensus.cc:348] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.523517 23760 raft_consensus.cc:348] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.523851 23760 raft_consensus.cc:374] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:05.523926 23762 raft_consensus.cc:374] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:05.524000 23595 heartbeater.cc:503] Master 127.18.72.254:46123 was elected leader, sending a full tablet report...
I20260811 07:39:05.524003 23760 raft_consensus.cc:729] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb9f31fecf74f8e98248171a4cb5532, State: Initialized, Role: FOLLOWER
I20260811 07:39:05.524037 23762 raft_consensus.cc:729] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa49d29c2f164e92a4c41788ccaa18d9, State: Initialized, Role: FOLLOWER
I20260811 07:39:05.524391 23761 ts_tablet_manager.cc:1428] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 07:39:05.524613 23762 consensus_queue.cc:260] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.524577 23760 consensus_queue.cc:260] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.525743 23760 ts_tablet_manager.cc:1428] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260811 07:39:05.525733 23762 ts_tablet_manager.cc:1428] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 07:39:05.526116 23741 heartbeater.cc:503] Master 127.18.72.254:46123 was elected leader, sending a full tablet report...
I20260811 07:39:05.527143 23444 heartbeater.cc:503] Master 127.18.72.254:46123 was elected leader, sending a full tablet report...
I20260811 07:39:05.595620 23766 raft_consensus.cc:482] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:05.595863 23766 raft_consensus.cc:504] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.597294 23766 leader_election.cc:288] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa49d29c2f164e92a4c41788ccaa18d9 (127.18.72.195:35307), 7cb9f31fecf74f8e98248171a4cb5532 (127.18.72.193:38393)
I20260811 07:39:05.601472 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.597585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:35307 (local address 127.18.72.194:42245)
0811 07:39:05.597736 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.597754 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:05.597864 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:05.598289 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:05.598292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:05.598318 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:05.598453 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.598462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.599304 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.599310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.600385 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.600396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.601060 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.601064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.601168 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.601192 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 07:39:05.601264 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:39:05.601340 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":58}
I20260811 07:39:05.601501 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.597730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:42245 (local address 127.18.72.195:35307)
0811 07:39:05.598029 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.598034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.598049 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:39:05.598106 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.598110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.598172 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.598254 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:05.598573 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.599182 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.600506 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.600937 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.601056 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.601116 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.601301 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.601379 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260811 07:39:05.601539 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.597590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:38393 (local address 127.18.72.194:44063)
0811 07:39:05.598104 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.598116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:05.598227 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:05.598488 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:05.598491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:05.598528 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:05.598654 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.598661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.599386 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.599389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.600412 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:05.600423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.601111 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.601115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:05.601222 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.601260 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 07:39:05.601342 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 07:39:05.601406 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":146,"thread_start_us":145,"threads_started":1}
I20260811 07:39:05.601567 23759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.597822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:44063 (local address 127.18.72.193:38393)
0811 07:39:05.597961 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.597967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.598196 (+ 229us) server_negotiation.cc:407] Connection header received
0811 07:39:05.598325 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.598328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.598378 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.598465 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:05.598764 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.599256 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.600558 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.600960 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.601097 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.601150 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.601373 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.601443 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:39:05.602425 23689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c423a0c9555a492d90c24dea5097d8a9" candidate_uuid: "d8849ef02bc9485a8114397bb76795c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" is_pre_election: true
I20260811 07:39:05.602451 23394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c423a0c9555a492d90c24dea5097d8a9" candidate_uuid: "d8849ef02bc9485a8114397bb76795c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cb9f31fecf74f8e98248171a4cb5532" is_pre_election: true
I20260811 07:39:05.602751 23689 raft_consensus.cc:2454] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8849ef02bc9485a8114397bb76795c8 in term 0.
I20260811 07:39:05.602751 23394 raft_consensus.cc:2454] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8849ef02bc9485a8114397bb76795c8 in term 0.
I20260811 07:39:05.603219 23469 leader_election.cc:302] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cb9f31fecf74f8e98248171a4cb5532, d8849ef02bc9485a8114397bb76795c8; no voters:
I20260811 07:39:05.603439 23766 raft_consensus.cc:2781] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:05.603554 23766 raft_consensus.cc:482] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:05.603613 23766 raft_consensus.cc:3037] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:05.604773 23766 raft_consensus.cc:504] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.605124 23766 leader_election.cc:288] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [CANDIDATE]: Term 1 election: Requested vote from peers fa49d29c2f164e92a4c41788ccaa18d9 (127.18.72.195:35307), 7cb9f31fecf74f8e98248171a4cb5532 (127.18.72.193:38393)
I20260811 07:39:05.605506 23689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c423a0c9555a492d90c24dea5097d8a9" candidate_uuid: "d8849ef02bc9485a8114397bb76795c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa49d29c2f164e92a4c41788ccaa18d9"
I20260811 07:39:05.605506 23394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c423a0c9555a492d90c24dea5097d8a9" candidate_uuid: "d8849ef02bc9485a8114397bb76795c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cb9f31fecf74f8e98248171a4cb5532"
I20260811 07:39:05.605638 23689 raft_consensus.cc:3037] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:05.605638 23394 raft_consensus.cc:3037] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:05.606935 23689 raft_consensus.cc:2454] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8849ef02bc9485a8114397bb76795c8 in term 1.
I20260811 07:39:05.606935 23394 raft_consensus.cc:2454] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8849ef02bc9485a8114397bb76795c8 in term 1.
I20260811 07:39:05.607252 23469 leader_election.cc:302] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7cb9f31fecf74f8e98248171a4cb5532, d8849ef02bc9485a8114397bb76795c8, fa49d29c2f164e92a4c41788ccaa18d9; no voters:
I20260811 07:39:05.607453 23766 raft_consensus.cc:2781] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:05.607605 23766 raft_consensus.cc:686] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 1 LEADER]: Becoming Leader. State: Replica: d8849ef02bc9485a8114397bb76795c8, State: Running, Role: LEADER
I20260811 07:39:05.607993 23766 consensus_queue.cc:237] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } }
I20260811 07:39:05.611613 23233 catalog_manager.cc:5495] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8849ef02bc9485a8114397bb76795c8 (127.18.72.194). New cstate: current_term: 1 leader_uuid: "d8849ef02bc9485a8114397bb76795c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8849ef02bc9485a8114397bb76795c8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 } health_report { overall_health: UNKNOWN } } }
I20260811 07:39:05.652068 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.648304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.18.72.194:39937)
0811 07:39:05.648461 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.648468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.648485 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:39:05.648574 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.648576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.648629 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.648743 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:05.649034 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.649543 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.650398 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:05.650713 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:05.650835 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:05.651165 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:05.651614 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:05.651757 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.651823 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.651880 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 07:39:05.656265 23445 tablet.cc:2369] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:05.658387 23689 raft_consensus.cc:1261] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 1 FOLLOWER]: Refusing update from remote peer d8849ef02bc9485a8114397bb76795c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:05.658386 23394 raft_consensus.cc:1261] T c423a0c9555a492d90c24dea5097d8a9 P 7cb9f31fecf74f8e98248171a4cb5532 [term 1 FOLLOWER]: Refusing update from remote peer d8849ef02bc9485a8114397bb76795c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:05.659127 23770 consensus_queue.cc:1035] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cb9f31fecf74f8e98248171a4cb5532" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:05.659322 23766 consensus_queue.cc:1035] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa49d29c2f164e92a4c41788ccaa18d9" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:05.669060 23775 mvcc.cc:205] Tried to move back new op lower bound from 7317233441408413696 to 7317233441213497344. Current Snapshot: MvccSnapshot[applied={T|T < 7317233441408413696}]
I20260811 07:39:05.676307 23235 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:05.676466 23235 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:05.678943 23235 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21983cf91c9b4407bbddef8ba5af0176: Sending DeleteTablet for 3 replicas of tablet c423a0c9555a492d90c24dea5097d8a9
I20260811 07:39:05.679718 23668 tablet_service.cc:1514] Processing DeleteTablet for tablet c423a0c9555a492d90c24dea5097d8a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:05 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:48198
I20260811 07:39:05.679801 23372 tablet_service.cc:1514] Processing DeleteTablet for tablet c423a0c9555a492d90c24dea5097d8a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:05 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:44498
I20260811 07:39:05.680214 23783 tablet_replica.cc:330] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: stopping tablet replica
I20260811 07:39:05.680522 23783 raft_consensus.cc:2229] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:05.680495 23523 tablet_service.cc:1514] Processing DeleteTablet for tablet c423a0c9555a492d90c24dea5097d8a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:05 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:48810
I20260811 07:39:05.680850 23783 raft_consensus.cc:2258] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:05.680922 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 23305
I20260811 07:39:05.683645 23785 tablet_replica.cc:330] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: stopping tablet replica
I20260811 07:39:05.683946 23785 raft_consensus.cc:2229] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:39:05.684252 23783 ts_tablet_manager.cc:1905] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:05.684336 23785 raft_consensus.cc:2258] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:05.685664 23785 ts_tablet_manager.cc:1905] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:05.686998 23783 ts_tablet_manager.cc:1918] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:05.687090 23783 log.cc:1196] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/c423a0c9555a492d90c24dea5097d8a9
I20260811 07:39:05.687433 23783 ts_tablet_manager.cc:1939] T c423a0c9555a492d90c24dea5097d8a9 P fa49d29c2f164e92a4c41788ccaa18d9: Deleting consensus metadata
I20260811 07:39:05.688382 23785 ts_tablet_manager.cc:1918] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:05.688485 23785 log.cc:1196] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/c423a0c9555a492d90c24dea5097d8a9
I20260811 07:39:05.688700 23218 catalog_manager.cc:4841] TS fa49d29c2f164e92a4c41788ccaa18d9 (127.18.72.195:35307): tablet c423a0c9555a492d90c24dea5097d8a9 (table test-table [id=8f3883cd9e834bdf9238118f0240781c]) successfully deleted
W20260811 07:39:05.694343 23217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:38393 (error 108)
I20260811 07:39:05.694691 23785 ts_tablet_manager.cc:1939] T c423a0c9555a492d90c24dea5097d8a9 P d8849ef02bc9485a8114397bb76795c8: Deleting consensus metadata
I20260811 07:39:05.695441 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.695116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:38393 (local address 127.0.0.1:44512)
0811 07:39:05.695245 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.695324 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:38393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 07:39:05.695519 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 23450
W20260811 07:39:05.695611 23217 catalog_manager.cc:4584] TS 7cb9f31fecf74f8e98248171a4cb5532 (127.18.72.193:38393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c423a0c9555a492d90c24dea5097d8a9: Network error: Client connection negotiation failed: client connection to 127.18.72.193:38393: connect: Connection refused (error 111)
I20260811 07:39:05.703315 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 23600
I20260811 07:39:05.705005 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.703451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:39937 (local address 127.0.0.1:48826)
0811 07:39:05.704823 (+ 1372us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:05.704893 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.194:39937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1265}
I20260811 07:39:05.710588 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 23201
2026-08-11T07:39:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2525 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T07:39:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:05Z Disabled control of system clock
I20260811 07:39:05.742892 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:44577
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:44577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 07:39:05.856784 23791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:05.857044 23791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:05.857128 23791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:05.861284 23791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:05.861346 23791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:05.861361 23791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:05.861400 23791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:05.861433 23791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:05.866200 23791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:44577
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:44577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:05.867404 23791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:05.869004 23791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:05.877696 23796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:05.877737 23798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:05.877709 23800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:05.877862 23791 server_base.cc:1034] running on GCE node
I20260811 07:39:05.878599 23791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:05.879726 23791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:05.880959 23791 hybrid_clock.cc:648] HybridClock initialized: now 1786433945880928 us; error 47 us; skew 500 ppm
I20260811 07:39:05.882895 23791 webserver.cc:460] Webserver started at http://127.18.72.254:44931/ using document root <none> and password file <none>
I20260811 07:39:05.883513 23791 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:05.883610 23791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:05.883860 23791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:05.886003 23791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "34117c2e16264f51942a1f3935f55440"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "bf46ea0a062a8a1c4346cfbe05bd18d4"
server_key_iv: "4579d82af6780cd4dce2e2947119e7ba"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.886582 23791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "34117c2e16264f51942a1f3935f55440"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "bf46ea0a062a8a1c4346cfbe05bd18d4"
server_key_iv: "4579d82af6780cd4dce2e2947119e7ba"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.890470 23791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 07:39:05.892992 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:05.894219 23791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:05.894371 23791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "34117c2e16264f51942a1f3935f55440"
format_stamp: "Formatted at 2026-08-11 07:39:05 on dist-test-slave-zkpd"
server_key: "bf46ea0a062a8a1c4346cfbe05bd18d4"
server_key_iv: "4579d82af6780cd4dce2e2947119e7ba"
server_key_version: "encryptionkey@0"
I20260811 07:39:05.894488 23791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:05.899165 23791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:05.899763 23791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:05.900002 23791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:05.906716 23791 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:44577
I20260811 07:39:05.906728 23865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:44577 every 8 connection(s)
I20260811 07:39:05.907624 23791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:05.908735 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 23791
I20260811 07:39:05.908885 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:05.909186 18723 external_mini_cluster.cc:1428] Setting key 956cc0202c00a036696ce5942f9732fe
I20260811 07:39:05.911239 23866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:05.914803 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:05.910627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.18.72.254:44577)
0811 07:39:05.911251 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:05.911266 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:05.911307 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:39:05.912083 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:05.912101 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:05.912443 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:05.912769 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:05.913593 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:05.913597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:05.913609 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:05.913619 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:05.913672 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:05.913700 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:05.913847 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:05.913919 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 07:39:05.914159 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":138,"threads_started":1}
I20260811 07:39:05.917399 23866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440: Bootstrap starting.
I20260811 07:39:05.919534 23866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:05.920393 23866 log.cc:824] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:05.922223 23866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440: No bootstrap required, opened a new log
I20260811 07:39:05.925755 23866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34117c2e16264f51942a1f3935f55440" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44577 } }
I20260811 07:39:05.925971 23866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:05.926061 23866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34117c2e16264f51942a1f3935f55440, State: Initialized, Role: FOLLOWER
I20260811 07:39:05.926586 23866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34117c2e16264f51942a1f3935f55440" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44577 } }
I20260811 07:39:05.926775 23866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:05.926856 23866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:05.926960 23866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:05.927817 23866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34117c2e16264f51942a1f3935f55440" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44577 } }
I20260811 07:39:05.928203 23866 leader_election.cc:302] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34117c2e16264f51942a1f3935f55440; no voters:
I20260811 07:39:05.928515 23866 leader_election.cc:288] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:05.928624 23871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:05.928893 23871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [term 1 LEADER]: Becoming Leader. State: Replica: 34117c2e16264f51942a1f3935f55440, State: Running, Role: LEADER
I20260811 07:39:05.929246 23871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34117c2e16264f51942a1f3935f55440" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44577 } }
I20260811 07:39:05.929808 23866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:05.930922 23873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34117c2e16264f51942a1f3935f55440. Latest consensus state: current_term: 1 leader_uuid: "34117c2e16264f51942a1f3935f55440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34117c2e16264f51942a1f3935f55440" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44577 } } }
I20260811 07:39:05.931048 23873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:05.931337 23872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34117c2e16264f51942a1f3935f55440" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34117c2e16264f51942a1f3935f55440" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44577 } } }
I20260811 07:39:05.931443 23872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:05.931568 23880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:05.935374 23880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:05.940955 23880 catalog_manager.cc:1347] Generated new cluster ID: 26bbba2520894e6b82394c66dce7fec5
I20260811 07:39:05.941025 23880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:05.977782 23880 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:05.978173 23880 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:05.984680 23880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440: Generated new TSK 0
I20260811 07:39:05.985387 23880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:39:06.000728 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44577
--builtin_ntp_servers=127.18.72.212:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 07:39:06.116113 23892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:06.116381 23892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:06.116422 23892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:06.120442 23892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:06.120498 23892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:06.120611 23892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:06.125308 23892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:06.126447 23892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:06.127945 23892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:06.137795 23897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:06.137836 23900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:06.137809 23898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:06.138077 23892 server_base.cc:1034] running on GCE node
I20260811 07:39:06.138584 23892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:06.139209 23892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:06.140393 23892 hybrid_clock.cc:648] HybridClock initialized: now 1786433946140375 us; error 26 us; skew 500 ppm
I20260811 07:39:06.142223 23892 webserver.cc:460] Webserver started at http://127.18.72.193:44759/ using document root <none> and password file <none>
I20260811 07:39:06.142818 23892 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:06.142916 23892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:06.143162 23892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:06.145362 23892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "fc9f561595624f49841cafef920e40df"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "1a69047032ac56629e505023b2de456a"
server_key_iv: "fe8a69d93d364a661560f951e27d5aab"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.145951 23892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "fc9f561595624f49841cafef920e40df"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "1a69047032ac56629e505023b2de456a"
server_key_iv: "fe8a69d93d364a661560f951e27d5aab"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.149757 23892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:39:06.152532 23908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.153719 23892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:06.153853 23892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "fc9f561595624f49841cafef920e40df"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "1a69047032ac56629e505023b2de456a"
server_key_iv: "fe8a69d93d364a661560f951e27d5aab"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.153968 23892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:06.158640 23892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:06.159245 23892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:06.159437 23892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:06.160019 23892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:06.161113 23892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:06.161182 23892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.161245 23892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:06.161295 23892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.170063 23892 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:37219
I20260811 07:39:06.170089 24031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:37219 every 8 connection(s)
I20260811 07:39:06.171041 23892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:06.177510 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.173077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:53943 (local address 127.18.72.254:44577)
0811 07:39:06.173240 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.173245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.174452 (+ 1207us) server_negotiation.cc:407] Connection header received
0811 07:39:06.175774 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.175778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.175819 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.175950 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.177119 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.177121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.177123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.177132 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.177148 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.177156 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.177231 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.177362 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.177400 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:39:06.178114 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 23892
I20260811 07:39:06.178244 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:06.178571 18723 external_mini_cluster.cc:1428] Setting key 30432e5a18867c48b47a7a0998f46f40
I20260811 07:39:06.178678 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.173497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44577 (local address 127.18.72.193:53943)
0811 07:39:06.174270 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.174322 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:06.175574 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:06.176148 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:06.176155 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:06.176244 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:06.176867 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:06.176869 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:06.176910 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:06.176980 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:06.177231 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:06.177239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:06.177243 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:39:06.177377 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 07:39:06.177607 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":159,"threads_started":1}
I20260811 07:39:06.180001 24032 heartbeater.cc:347] Connected to a master server at 127.18.72.254:44577
I20260811 07:39:06.180254 24032 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:06.180739 24032 heartbeater.cc:511] Master 127.18.72.254:44577 requested a full tablet report, sending...
I20260811 07:39:06.181082 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44577
--builtin_ntp_servers=127.18.72.212:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 07:39:06.182521 23825 ts_manager.cc:194] Registered new tserver with Master: fc9f561595624f49841cafef920e40df (127.18.72.193:37219)
W20260811 07:39:06.296799 24035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:06.297081 24035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:06.297122 24035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:06.301601 24035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:06.301662 24035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:06.301728 24035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:06.306474 24035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:06.307542 24035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:06.309113 24035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:39:06.319389 24035 server_base.cc:1034] running on GCE node
W20260811 07:39:06.319350 24040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:06.319375 24043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:06.319351 24041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:06.320082 24035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:06.320719 24035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:06.321909 24035 hybrid_clock.cc:648] HybridClock initialized: now 1786433946321892 us; error 27 us; skew 500 ppm
I20260811 07:39:06.323853 24035 webserver.cc:460] Webserver started at http://127.18.72.194:39465/ using document root <none> and password file <none>
I20260811 07:39:06.324504 24035 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:06.324604 24035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:06.324853 24035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:06.326907 24035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "639ff623b542437e8d097b6923d9cba8"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "13c1eb3fc8a97b22756e334ef0ee315c"
server_key_iv: "940ea041bf4dae1eb40c22586828fa21"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.327473 24035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "639ff623b542437e8d097b6923d9cba8"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "13c1eb3fc8a97b22756e334ef0ee315c"
server_key_iv: "940ea041bf4dae1eb40c22586828fa21"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.331385 24035 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 07:39:06.334021 24050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.335237 24035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:06.335350 24035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "639ff623b542437e8d097b6923d9cba8"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "13c1eb3fc8a97b22756e334ef0ee315c"
server_key_iv: "940ea041bf4dae1eb40c22586828fa21"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.335438 24035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:06.339803 24035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:06.340436 24035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:06.340574 24035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:06.341053 24035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:06.341904 24035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:06.341949 24035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.341986 24035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:06.342001 24035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.350602 24035 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:38095
I20260811 07:39:06.350690 24176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:38095 every 8 connection(s)
I20260811 07:39:06.351562 24035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:06.358167 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.353578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:40245 (local address 127.18.72.254:44577)
0811 07:39:06.353758 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.353762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.354858 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 07:39:06.356327 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.356330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.356378 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.356513 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.357629 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.357633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.357636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.357649 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.357671 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.357688 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.357819 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.357975 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.358025 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:39:06.359180 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 24035
I20260811 07:39:06.359195 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.354012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44577 (local address 127.18.72.194:40245)
0811 07:39:06.354691 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.354739 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:06.356087 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:06.356698 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:06.356705 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:06.356800 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:06.357359 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:06.357361 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:06.357403 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:06.357475 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:06.357812 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:06.357821 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:06.357830 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 07:39:06.358025 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 07:39:06.358301 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":116,"threads_started":1}
I20260811 07:39:06.359314 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:06.359642 18723 external_mini_cluster.cc:1428] Setting key 39ebc115e28351085f441964dac41b76
I20260811 07:39:06.360543 24177 heartbeater.cc:347] Connected to a master server at 127.18.72.254:44577
I20260811 07:39:06.360812 24177 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:06.361326 24177 heartbeater.cc:511] Master 127.18.72.254:44577 requested a full tablet report, sending...
I20260811 07:39:06.361866 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44577
--builtin_ntp_servers=127.18.72.212:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 07:39:06.362419 23825 ts_manager.cc:194] Registered new tserver with Master: 639ff623b542437e8d097b6923d9cba8 (127.18.72.194:38095)
W20260811 07:39:06.477540 24180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:06.477783 24180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:06.477823 24180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:06.481851 24180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:06.481904 24180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:06.481995 24180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:06.487097 24180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:44577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:06.488262 24180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:06.489674 24180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:06.499277 24188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:06.499270 24186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:06.499348 24185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:06.499449 24180 server_base.cc:1034] running on GCE node
I20260811 07:39:06.499938 24180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:06.500545 24180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:06.501703 24180 hybrid_clock.cc:648] HybridClock initialized: now 1786433946501686 us; error 27 us; skew 500 ppm
I20260811 07:39:06.503523 24180 webserver.cc:460] Webserver started at http://127.18.72.195:35129/ using document root <none> and password file <none>
I20260811 07:39:06.504207 24180 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:06.504272 24180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:06.504516 24180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:06.506515 24180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "d8f6ffbf937e4701a323ecc410430aa6"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "88acec5ad1260cb95ec3c1aca3079426"
server_key_iv: "4f68be50d1749fec26a374c01879939f"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.507088 24180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "d8f6ffbf937e4701a323ecc410430aa6"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "88acec5ad1260cb95ec3c1aca3079426"
server_key_iv: "4f68be50d1749fec26a374c01879939f"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.510742 24180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:39:06.513327 24194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.514575 24180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:06.514714 24180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "d8f6ffbf937e4701a323ecc410430aa6"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "88acec5ad1260cb95ec3c1aca3079426"
server_key_iv: "4f68be50d1749fec26a374c01879939f"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.514842 24180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:06.519654 24180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:06.520327 24180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:06.520516 24180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:06.521072 24180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:06.521992 24180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:06.522071 24180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.522140 24180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:06.522173 24180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.531307 24180 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:43655
I20260811 07:39:06.531363 24321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:43655 every 8 connection(s)
I20260811 07:39:06.532359 24180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:06.538522 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.534236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:37461 (local address 127.18.72.254:44577)
0811 07:39:06.534392 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.534396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.535527 (+ 1131us) server_negotiation.cc:407] Connection header received
0811 07:39:06.536859 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.536862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.536901 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.536977 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.538086 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.538089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.538091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.538100 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.538117 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.538124 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.538198 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.538322 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.538359 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:39:06.539228 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 24180
I20260811 07:39:06.539366 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:06.539490 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.534717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44577 (local address 127.18.72.195:37461)
0811 07:39:06.535379 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.535419 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:06.536667 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:06.537140 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:06.537147 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:06.537234 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:06.537848 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:06.537849 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:06.537889 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:06.537959 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:06.538202 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:06.538207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:06.538211 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:39:06.538340 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 07:39:06.538530 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":112,"threads_started":1}
I20260811 07:39:06.539721 18723 external_mini_cluster.cc:1428] Setting key a286c670fb0c269374e9eb86892dbe0c
I20260811 07:39:06.540730 24322 heartbeater.cc:347] Connected to a master server at 127.18.72.254:44577
I20260811 07:39:06.541108 24322 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:06.541568 24322 heartbeater.cc:511] Master 127.18.72.254:44577 requested a full tablet report, sending...
I20260811 07:39:06.542595 23825 ts_manager.cc:194] Registered new tserver with Master: d8f6ffbf937e4701a323ecc410430aa6 (127.18.72.195:43655)
I20260811 07:39:06.553798 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:39:06.557576 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.555771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.18.72.254:44577)
0811 07:39:06.555951 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.555956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.556135 (+ 179us) server_negotiation.cc:407] Connection header received
0811 07:39:06.556228 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.556230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.556266 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.556334 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.557103 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.557106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.557108 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.557118 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.557136 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.557144 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.557286 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.557382 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.557417 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:39:06.560372 23825 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:39:06.563632 23825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:06.574966 24335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.572270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37219 (local address 127.0.0.1:39048)
0811 07:39:06.573133 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.573148 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:06.573227 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:06.573702 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:06.573706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:06.573724 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:06.574148 (+ 424us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:06.574150 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:06.574161 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:06.574191 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:06.574617 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:06.574627 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:06.574634 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:39:06.574733 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 07:39:06.574809 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":106,"threads_started":1}
I20260811 07:39:06.575140 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.571929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:38095 (local address 127.0.0.1:53848)
0811 07:39:06.572297 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.572350 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:06.572447 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:06.573242 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:06.573249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:06.573332 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:06.573949 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:06.573951 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:06.573975 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:06.574053 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:06.574825 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:06.574829 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:06.574833 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:39:06.574917 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 07:39:06.574955 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260811 07:39:06.575165 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.572027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53848 (local address 127.18.72.194:38095)
0811 07:39:06.572729 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.572741 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.572769 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:39:06.572845 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.572852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.573065 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.573616 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.574606 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.574611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.574623 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.574634 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.574671 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.574699 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.574845 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.574938 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.575015 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":141,"threads_started":1}
I20260811 07:39:06.575712 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.572715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.18.72.193:37219)
0811 07:39:06.573274 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.573302 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.573324 (+ 22us) server_negotiation.cc:407] Connection header received
0811 07:39:06.573375 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.573380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.573557 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.573683 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.574385 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.574392 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.574404 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.574415 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.574472 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.574501 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.575425 (+ 924us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.575474 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.575541 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":97,"threads_started":1}
I20260811 07:39:06.576766 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.572634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:43655 (local address 127.0.0.1:33302)
0811 07:39:06.573447 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.573460 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:06.574148 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:06.574580 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:06.574583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:06.574597 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:06.576165 (+ 1568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:06.576167 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:06.576177 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:06.576204 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:06.576529 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:06.576532 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:06.576535 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:39:06.576613 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:39:06.576649 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":92,"threads_started":1}
I20260811 07:39:06.577142 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.573068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33302 (local address 127.18.72.195:43655)
0811 07:39:06.573463 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.573478 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.573525 (+ 47us) server_negotiation.cc:407] Connection header received
0811 07:39:06.574298 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.574304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.574453 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.574597 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.576348 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.576353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.576361 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.576371 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.576406 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.576427 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.576542 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.576692 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.576756 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":118,"threads_started":1}
I20260811 07:39:06.578574 23959 tablet_service.cc:1467] Processing CreateTablet for tablet 07ab05cd66d6433c8cfce8573c7272a3 (DEFAULT_TABLE table=test-table [id=2b103846ebf144cd983e89c5ff5835a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:06.578574 24103 tablet_service.cc:1467] Processing CreateTablet for tablet 07ab05cd66d6433c8cfce8573c7272a3 (DEFAULT_TABLE table=test-table [id=2b103846ebf144cd983e89c5ff5835a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:06.579754 24103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ab05cd66d6433c8cfce8573c7272a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:06.579754 23959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ab05cd66d6433c8cfce8573c7272a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:06.579651 24246 tablet_service.cc:1467] Processing CreateTablet for tablet 07ab05cd66d6433c8cfce8573c7272a3 (DEFAULT_TABLE table=test-table [id=2b103846ebf144cd983e89c5ff5835a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:06.580603 24246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07ab05cd66d6433c8cfce8573c7272a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:06.586534 24339 tablet_bootstrap.cc:492] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: Bootstrap starting.
I20260811 07:39:06.587853 24340 tablet_bootstrap.cc:492] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df: Bootstrap starting.
I20260811 07:39:06.588238 24341 tablet_bootstrap.cc:492] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Bootstrap starting.
I20260811 07:39:06.589457 24339 tablet_bootstrap.cc:654] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:06.590044 24340 tablet_bootstrap.cc:654] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:06.590096 24341 tablet_bootstrap.cc:654] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:06.590251 24339 log.cc:824] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:06.590916 24340 log.cc:824] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:06.590916 24341 log.cc:824] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:06.592249 24339 tablet_bootstrap.cc:492] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: No bootstrap required, opened a new log
I20260811 07:39:06.592463 24339 ts_tablet_manager.cc:1397] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:39:06.592918 24341 tablet_bootstrap.cc:492] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: No bootstrap required, opened a new log
I20260811 07:39:06.592918 24340 tablet_bootstrap.cc:492] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df: No bootstrap required, opened a new log
I20260811 07:39:06.593101 24340 ts_tablet_manager.cc:1397] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:39:06.593101 24341 ts_tablet_manager.cc:1397] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:39:06.596421 24339 raft_consensus.cc:348] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.596668 24339 raft_consensus.cc:374] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:06.596722 24339 raft_consensus.cc:729] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f6ffbf937e4701a323ecc410430aa6, State: Initialized, Role: FOLLOWER
I20260811 07:39:06.597193 24339 consensus_queue.cc:260] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.597736 24341 raft_consensus.cc:348] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.597966 24341 raft_consensus.cc:374] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:06.597978 24322 heartbeater.cc:503] Master 127.18.72.254:44577 was elected leader, sending a full tablet report...
I20260811 07:39:06.598039 24341 raft_consensus.cc:729] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ff623b542437e8d097b6923d9cba8, State: Initialized, Role: FOLLOWER
I20260811 07:39:06.598384 24339 ts_tablet_manager.cc:1428] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:39:06.598459 24341 consensus_queue.cc:260] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.597680 24340 raft_consensus.cc:348] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.598645 24340 raft_consensus.cc:374] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:06.598709 24340 raft_consensus.cc:729] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9f561595624f49841cafef920e40df, State: Initialized, Role: FOLLOWER
I20260811 07:39:06.599123 24340 consensus_queue.cc:260] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.599433 24341 ts_tablet_manager.cc:1428] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 07:39:06.599601 24177 heartbeater.cc:503] Master 127.18.72.254:44577 was elected leader, sending a full tablet report...
I20260811 07:39:06.599852 24032 heartbeater.cc:503] Master 127.18.72.254:44577 was elected leader, sending a full tablet report...
I20260811 07:39:06.600071 24340 ts_tablet_manager.cc:1428] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 07:39:06.602552 24178 tablet.cc:2369] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:39:06.672317 24033 tablet.cc:2369] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:06.692704 24348 raft_consensus.cc:482] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:06.692981 24348 raft_consensus.cc:504] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.694186 24348 leader_election.cc:288] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ff623b542437e8d097b6923d9cba8 (127.18.72.194:38095), d8f6ffbf937e4701a323ecc410430aa6 (127.18.72.195:43655)
I20260811 07:39:06.696390 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.694541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:45317 (local address 127.18.72.194:38095)
0811 07:39:06.694702 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.694708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.694797 (+ 89us) server_negotiation.cc:407] Connection header received
0811 07:39:06.694974 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.694977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.695019 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.695077 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.695962 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.695966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.695969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.695983 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.696009 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.696017 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.696095 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.696223 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.696266 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:39:06.696412 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.694393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:38095 (local address 127.18.72.193:45317)
0811 07:39:06.694713 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.694730 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:06.694841 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:06.695215 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:06.695221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:06.695238 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:06.695776 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:06.695779 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:06.695793 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:06.695829 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:06.696110 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:06.696113 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:06.696115 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 07:39:06.696218 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 07:39:06.696259 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":120,"threads_started":1}
I20260811 07:39:06.696733 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.694984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:58125 (local address 127.18.72.195:43655)
0811 07:39:06.695116 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.695121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.695164 (+ 43us) server_negotiation.cc:407] Connection header received
0811 07:39:06.695252 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.695255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.695294 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.695385 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.696309 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.696312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.696314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.696326 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.696344 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.696355 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.696458 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.696557 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.696596 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:39:06.697096 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.694822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:43655 (local address 127.18.72.193:58125)
0811 07:39:06.695100 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.695112 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:06.695184 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:06.695426 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:06.695431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:06.695448 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:06.696160 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:06.696161 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:06.696170 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:06.696191 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:06.696444 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:06.696447 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:06.696450 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 07:39:06.696916 (+ 466us) client_negotiation.cc:240] Negotiation successful
0811 07:39:06.696957 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
I20260811 07:39:06.697253 24125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ab05cd66d6433c8cfce8573c7272a3" candidate_uuid: "fc9f561595624f49841cafef920e40df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ff623b542437e8d097b6923d9cba8" is_pre_election: true
I20260811 07:39:06.697561 24125 raft_consensus.cc:2454] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc9f561595624f49841cafef920e40df in term 0.
I20260811 07:39:06.698091 23910 leader_election.cc:302] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 639ff623b542437e8d097b6923d9cba8, fc9f561595624f49841cafef920e40df; no voters:
I20260811 07:39:06.698336 24270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ab05cd66d6433c8cfce8573c7272a3" candidate_uuid: "fc9f561595624f49841cafef920e40df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f6ffbf937e4701a323ecc410430aa6" is_pre_election: true
I20260811 07:39:06.698390 24348 raft_consensus.cc:2781] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:06.698487 24348 raft_consensus.cc:482] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:06.698557 24348 raft_consensus.cc:3037] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:06.698629 24270 raft_consensus.cc:2454] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc9f561595624f49841cafef920e40df in term 0.
I20260811 07:39:06.699740 24348 raft_consensus.cc:504] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.700100 24348 leader_election.cc:288] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [CANDIDATE]: Term 1 election: Requested vote from peers 639ff623b542437e8d097b6923d9cba8 (127.18.72.194:38095), d8f6ffbf937e4701a323ecc410430aa6 (127.18.72.195:43655)
I20260811 07:39:06.700598 24270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ab05cd66d6433c8cfce8573c7272a3" candidate_uuid: "fc9f561595624f49841cafef920e40df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f6ffbf937e4701a323ecc410430aa6"
I20260811 07:39:06.700598 24125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07ab05cd66d6433c8cfce8573c7272a3" candidate_uuid: "fc9f561595624f49841cafef920e40df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ff623b542437e8d097b6923d9cba8"
I20260811 07:39:06.700742 24270 raft_consensus.cc:3037] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:06.700765 24125 raft_consensus.cc:3037] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:06.702169 24125 raft_consensus.cc:2454] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc9f561595624f49841cafef920e40df in term 1.
I20260811 07:39:06.702169 24270 raft_consensus.cc:2454] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc9f561595624f49841cafef920e40df in term 1.
I20260811 07:39:06.702450 23910 leader_election.cc:302] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 639ff623b542437e8d097b6923d9cba8, fc9f561595624f49841cafef920e40df; no voters:
I20260811 07:39:06.702626 24348 raft_consensus.cc:2781] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:06.702865 24348 raft_consensus.cc:686] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [term 1 LEADER]: Becoming Leader. State: Replica: fc9f561595624f49841cafef920e40df, State: Running, Role: LEADER
I20260811 07:39:06.703248 24348 consensus_queue.cc:237] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } }
I20260811 07:39:06.706818 23825 catalog_manager.cc:5495] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df reported cstate change: term changed from 0 to 1, leader changed from <none> to fc9f561595624f49841cafef920e40df (127.18.72.193). New cstate: current_term: 1 leader_uuid: "fc9f561595624f49841cafef920e40df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc9f561595624f49841cafef920e40df" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37219 } health_report { overall_health: HEALTHY } } }
I20260811 07:39:06.727949 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.726370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.18.72.193:37219)
0811 07:39:06.726503 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:06.726507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:06.726522 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:39:06.726683 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:06.726685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:06.726748 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:06.726859 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:06.727514 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:06.727517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:06.727520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:06.727532 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:06.727559 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:06.727567 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:06.727663 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:06.727736 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 07:39:06.727784 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 07:39:06.734081 24270 raft_consensus.cc:1261] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 1 FOLLOWER]: Refusing update from remote peer fc9f561595624f49841cafef920e40df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:06.734081 24125 raft_consensus.cc:1261] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 1 FOLLOWER]: Refusing update from remote peer fc9f561595624f49841cafef920e40df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:06.734905 24351 consensus_queue.cc:1035] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ff623b542437e8d097b6923d9cba8" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:06.735102 24348 consensus_queue.cc:1035] T 07ab05cd66d6433c8cfce8573c7272a3 P fc9f561595624f49841cafef920e40df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8f6ffbf937e4701a323ecc410430aa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:06.741791 24354 mvcc.cc:205] Tried to move back new op lower bound from 7317233445815357440 to 7317233445699895296. Current Snapshot: MvccSnapshot[applied={T|T < 7317233445815357440}]
I20260811 07:39:06.742655 24356 mvcc.cc:205] Tried to move back new op lower bound from 7317233445815357440 to 7317233445699895296. Current Snapshot: MvccSnapshot[applied={T|T < 7317233445815357440}]
I20260811 07:39:06.746003 24355 mvcc.cc:205] Tried to move back new op lower bound from 7317233445815357440 to 7317233445699895296. Current Snapshot: MvccSnapshot[applied={T|T < 7317233445815357440 or (T in {7317233445815357440})}]
I20260811 07:39:06.752625 23823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:06.752818 23823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:06.755158 23823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34117c2e16264f51942a1f3935f55440: Sending DeleteTablet for 3 replicas of tablet 07ab05cd66d6433c8cfce8573c7272a3
I20260811 07:39:06.756007 23959 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ab05cd66d6433c8cfce8573c7272a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:06 UTC) from {username='slave'} at 127.0.0.1:39048
I20260811 07:39:06.756055 24103 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ab05cd66d6433c8cfce8573c7272a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:06 UTC) from {username='slave'} at 127.0.0.1:53848
I20260811 07:39:06.756165 24246 tablet_service.cc:1514] Processing DeleteTablet for tablet 07ab05cd66d6433c8cfce8573c7272a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:06 UTC) from {username='slave'} at 127.0.0.1:33302
I20260811 07:39:06.756558 24368 tablet_replica.cc:330] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: stopping tablet replica
I20260811 07:39:06.756572 24369 tablet_replica.cc:330] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: stopping tablet replica
I20260811 07:39:06.756875 24368 raft_consensus.cc:2229] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:06.756875 24369 raft_consensus.cc:2229] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:06.756953 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 23892
I20260811 07:39:06.757227 24368 raft_consensus.cc:2258] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:06.757227 24369 raft_consensus.cc:2258] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:06.759358 24369 ts_tablet_manager.cc:1905] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:06.760947 24368 ts_tablet_manager.cc:1905] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:06.762223 24369 ts_tablet_manager.cc:1918] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:06.762328 24369 log.cc:1196] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/07ab05cd66d6433c8cfce8573c7272a3
I20260811 07:39:06.762661 24369 ts_tablet_manager.cc:1939] T 07ab05cd66d6433c8cfce8573c7272a3 P d8f6ffbf937e4701a323ecc410430aa6: Deleting consensus metadata
I20260811 07:39:06.763599 24368 ts_tablet_manager.cc:1918] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:06.763700 24368 log.cc:1196] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/07ab05cd66d6433c8cfce8573c7272a3
I20260811 07:39:06.763953 23810 catalog_manager.cc:4841] TS d8f6ffbf937e4701a323ecc410430aa6 (127.18.72.195:43655): tablet 07ab05cd66d6433c8cfce8573c7272a3 (table test-table [id=2b103846ebf144cd983e89c5ff5835a7]) successfully deleted
I20260811 07:39:06.764178 24368 ts_tablet_manager.cc:1939] T 07ab05cd66d6433c8cfce8573c7272a3 P 639ff623b542437e8d097b6923d9cba8: Deleting consensus metadata
I20260811 07:39:06.765142 23810 catalog_manager.cc:4841] TS 639ff623b542437e8d097b6923d9cba8 (127.18.72.194:38095): tablet 07ab05cd66d6433c8cfce8573c7272a3 (table test-table [id=2b103846ebf144cd983e89c5ff5835a7]) successfully deleted
W20260811 07:39:06.766340 23811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:37219 (error 108)
I20260811 07:39:06.767119 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 24035
I20260811 07:39:06.767351 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:06.767039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37219 (local address 127.0.0.1:39068)
0811 07:39:06.767172 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:06.767252 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:37219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 07:39:06.768107 23811 catalog_manager.cc:4584] TS fc9f561595624f49841cafef920e40df (127.18.72.193:37219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07ab05cd66d6433c8cfce8573c7272a3: Network error: Client connection negotiation failed: client connection to 127.18.72.193:37219: connect: Connection refused (error 111)
I20260811 07:39:06.776418 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 24180
I20260811 07:39:06.784752 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 23791
2026-08-11T07:39:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1075 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 07:39:06.809932 18723 mini_oidc.cc:152] Starting JWKS server
I20260811 07:39:06.811584 18723 webserver.cc:460] Webserver started at https://127.0.0.1:46051/ using document root <none> and password file <none>
I20260811 07:39:06.811786 18723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 07:39:06.812104 18723 webserver.cc:460] Webserver started at http://127.0.0.1:35159/ using document root <none> and password file <none>
2026-08-11T07:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:06Z Disabled control of system clock
I20260811 07:39:06.823366 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:44525
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:44525
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46051/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/testchainca.cert with env {}
W20260811 07:39:06.938848 24376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:06.939195 24376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:06.939239 24376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:06.943246 24376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:06.943311 24376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:06.943326 24376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:06.943342 24376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:06.943357 24376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 07:39:06.943373 24376 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 07:39:06.943387 24376 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46051/jwks.json
I20260811 07:39:06.948649 24376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:44525
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:44525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46051/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:06.949729 24376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:06.950745 24376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:06.959254 24387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:06.959270 24381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:06.959254 24384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:06.959424 24376 server_base.cc:1034] running on GCE node
I20260811 07:39:06.960217 24376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:06.961274 24376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:06.962441 24376 hybrid_clock.cc:648] HybridClock initialized: now 1786433946962407 us; error 43 us; skew 500 ppm
I20260811 07:39:06.964318 24376 webserver.cc:460] Webserver started at http://127.18.72.254:43667/ using document root <none> and password file <none>
I20260811 07:39:06.964874 24376 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:06.964933 24376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:06.965099 24376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:06.967106 24376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "93dbbe7087714d72b319122a3b6c30ba"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "6b124754cf7d8e5aa25a0818e1cf9101"
server_key_iv: "4d8878242452ce3c113913f450a1774f"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.967626 24376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "93dbbe7087714d72b319122a3b6c30ba"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "6b124754cf7d8e5aa25a0818e1cf9101"
server_key_iv: "4d8878242452ce3c113913f450a1774f"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.971400 24376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:39:06.973798 24393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:06.974982 24376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:06.975137 24376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "93dbbe7087714d72b319122a3b6c30ba"
format_stamp: "Formatted at 2026-08-11 07:39:06 on dist-test-slave-zkpd"
server_key: "6b124754cf7d8e5aa25a0818e1cf9101"
server_key_iv: "4d8878242452ce3c113913f450a1774f"
server_key_version: "encryptionkey@0"
I20260811 07:39:06.975250 24376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:06.996129 24376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:06.996815 24376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:06.997020 24376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:07.004040 24449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:44525 every 8 connection(s)
I20260811 07:39:07.004045 24376 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:44525
I20260811 07:39:07.005039 24376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:07.008385 24450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:07.010286 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 24376
I20260811 07:39:07.010437 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:07.010705 18723 external_mini_cluster.cc:1428] Setting key 41386d7ee557a47088702232cbe5bb2b
I20260811 07:39:07.015081 24450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba: Bootstrap starting.
I20260811 07:39:07.017726 24450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:07.018625 24450 log.cc:824] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:07.021018 24450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba: No bootstrap required, opened a new log
I20260811 07:39:07.021514 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:07.012149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.18.72.254:44525)
0811 07:39:07.012867 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:07.012890 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:07.012935 (+ 45us) server_negotiation.cc:407] Connection header received
0811 07:39:07.013956 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:07.013981 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:07.014500 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:07.014865 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:07.015309 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.016786 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.017691 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.018381 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.018562 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.019441 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:07.019446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:07.019459 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:07.020016 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:07.020084 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:07.020111 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:07.020205 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:07.020343 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 07:39:07.020603 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":142,"threads_started":1}
I20260811 07:39:07.024507 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:39:07.025408 24450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93dbbe7087714d72b319122a3b6c30ba" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44525 } }
I20260811 07:39:07.025669 24450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:07.025738 24450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93dbbe7087714d72b319122a3b6c30ba, State: Initialized, Role: FOLLOWER
I20260811 07:39:07.026199 24450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93dbbe7087714d72b319122a3b6c30ba" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44525 } }
I20260811 07:39:07.026321 24450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:07.026402 24450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:07.026505 24450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:07.027381 24450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93dbbe7087714d72b319122a3b6c30ba" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44525 } }
I20260811 07:39:07.027721 24450 leader_election.cc:302] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93dbbe7087714d72b319122a3b6c30ba; no voters:
I20260811 07:39:07.028043 24450 leader_election.cc:288] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:07.028162 24455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:07.028461 24455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [term 1 LEADER]: Becoming Leader. State: Replica: 93dbbe7087714d72b319122a3b6c30ba, State: Running, Role: LEADER
I20260811 07:39:07.028746 24455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93dbbe7087714d72b319122a3b6c30ba" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44525 } }
I20260811 07:39:07.029306 24450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:07.030493 24457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93dbbe7087714d72b319122a3b6c30ba. Latest consensus state: current_term: 1 leader_uuid: "93dbbe7087714d72b319122a3b6c30ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93dbbe7087714d72b319122a3b6c30ba" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44525 } } }
I20260811 07:39:07.030618 24457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:07.030962 24456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93dbbe7087714d72b319122a3b6c30ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93dbbe7087714d72b319122a3b6c30ba" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 44525 } } }
I20260811 07:39:07.031066 24456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:07.031148 24467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:07.033890 24467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:07.039202 24467 catalog_manager.cc:1347] Generated new cluster ID: 7e144ed6bd6e4238afe8e9e3e93d8322
I20260811 07:39:07.039271 24467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:07.063107 24467 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:07.064404 24467 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:07.077987 24467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93dbbe7087714d72b319122a3b6c30ba: Generated new TSK 0
I20260811 07:39:07.078842 24467 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 07:39:07.114861 24485 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 07:39:07.115054 24485 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 07:39:07.115803 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:07.096556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.18.72.254:44525)
0811 07:39:07.096691 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:07.096696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:07.096878 (+ 182us) server_negotiation.cc:407] Connection header received
0811 07:39:07.097065 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:07.097068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:07.097127 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:07.097235 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0811 07:39:07.097474 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.098317 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.099621 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.099992 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.100097 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.100396 (+ 299us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 07:39:07.115294 (+ 14898us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 07:39:07.115618 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54752: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 07:39:07.115943 24453 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54752: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 07:39:07.118072 24483 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:07.096446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:44525 (local address 127.0.0.1:54752)
0811 07:39:07.096796 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:07.096809 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:07.096917 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:07.097262 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:07.097264 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:07.097271 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0811 07:39:07.097379 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:07.097384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.098459 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.098465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:07.099511 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:07.099521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.100100 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.100103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:07.100170 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.100214 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 07:39:07.116188 (+ 15974us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 07:39:07.116459 (+ 271us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.72.254:44525: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":124,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.72.254:44525: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 07:39:07.118916 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 24376
2026-08-11T07:39:07Z chronyd exiting
I20260811 07:39:07.161729 18723 test_util.cc:175] -----------------------------------------------
I20260811 07:39:07.161818 18723 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786433920963264-18723-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (353 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 07:39:07.162972 18723 mini_oidc.cc:152] Starting JWKS server
I20260811 07:39:07.163830 18723 webserver.cc:460] Webserver started at https://127.0.0.1:37029/ using document root <none> and password file <none>
I20260811 07:39:07.163975 18723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 07:39:07.164204 18723 webserver.cc:460] Webserver started at http://127.0.0.1:37211/ using document root <none> and password file <none>
2026-08-11T07:39:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:07Z Disabled control of system clock
I20260811 07:39:07.174307 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41837
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:41837
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37029/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/testchainca.cert with env {}
W20260811 07:39:07.285917 24491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:07.286185 24491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:07.286226 24491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:07.290256 24491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:07.290318 24491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:07.290333 24491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:07.290350 24491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:07.290365 24491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 07:39:07.290381 24491 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 07:39:07.290395 24491 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37029/jwks.json
I20260811 07:39:07.295812 24491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:41837
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37029/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:07.296921 24491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:07.297950 24491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:39:07.306963 24491 server_base.cc:1034] running on GCE node
W20260811 07:39:07.306905 24502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:07.306905 24497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:07.306897 24499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:07.307873 24491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:07.309049 24491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:07.310261 24491 hybrid_clock.cc:648] HybridClock initialized: now 1786433947310244 us; error 31 us; skew 500 ppm
I20260811 07:39:07.312266 24491 webserver.cc:460] Webserver started at http://127.18.72.254:33689/ using document root <none> and password file <none>
I20260811 07:39:07.313117 24491 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:07.313223 24491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:07.313474 24491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:07.315565 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "7512b498705c4b26b508a69948cded53"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "9e1025bd581eb0e3bce79ff58d094746"
server_key_iv: "4b15f53455c1e2e481512d0e83b9045c"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.316159 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "7512b498705c4b26b508a69948cded53"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "9e1025bd581eb0e3bce79ff58d094746"
server_key_iv: "4b15f53455c1e2e481512d0e83b9045c"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.319738 24491 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:39:07.322129 24508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:07.323300 24491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:07.323438 24491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "7512b498705c4b26b508a69948cded53"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "9e1025bd581eb0e3bce79ff58d094746"
server_key_iv: "4b15f53455c1e2e481512d0e83b9045c"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.323550 24491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:07.343791 24491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:07.344515 24491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:07.344718 24491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:07.351979 24491 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:41837
I20260811 07:39:07.351999 24565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:41837 every 8 connection(s)
I20260811 07:39:07.352927 24491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:07.356241 24566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:07.360042 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 24491
I20260811 07:39:07.360190 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:07.360492 18723 external_mini_cluster.cc:1428] Setting key b43a0f9772349ac996cdb5dfa7236d6c
I20260811 07:39:07.362154 24566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53: Bootstrap starting.
I20260811 07:39:07.364719 24566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:07.365450 24566 log.cc:824] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:07.367612 24566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53: No bootstrap required, opened a new log
I20260811 07:39:07.369876 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:07.361905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.18.72.254:41837)
0811 07:39:07.362544 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:07.362560 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:07.362596 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:39:07.363347 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:07.363366 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:07.363752 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:07.364106 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:07.364501 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.365521 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.366663 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.367118 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.367236 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.368179 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:07.368185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:07.368198 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:07.368598 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:07.368652 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:07.368671 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:07.368745 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:07.368918 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 07:39:07.369195 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":139,"threads_started":1}
I20260811 07:39:07.371579 24566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7512b498705c4b26b508a69948cded53" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41837 } }
I20260811 07:39:07.371841 24566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:07.371919 24566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7512b498705c4b26b508a69948cded53, State: Initialized, Role: FOLLOWER
I20260811 07:39:07.372402 24566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7512b498705c4b26b508a69948cded53" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41837 } }
I20260811 07:39:07.372529 24566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:07.372605 24566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:07.372740 24566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:07.372865 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:39:07.373898 24566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7512b498705c4b26b508a69948cded53" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41837 } }
I20260811 07:39:07.374264 24566 leader_election.cc:302] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7512b498705c4b26b508a69948cded53; no voters:
I20260811 07:39:07.374624 24566 leader_election.cc:288] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:07.374707 24571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:07.374908 24571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [term 1 LEADER]: Becoming Leader. State: Replica: 7512b498705c4b26b508a69948cded53, State: Running, Role: LEADER
I20260811 07:39:07.375212 24571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7512b498705c4b26b508a69948cded53" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41837 } }
I20260811 07:39:07.375955 24566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:07.376977 24574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7512b498705c4b26b508a69948cded53. Latest consensus state: current_term: 1 leader_uuid: "7512b498705c4b26b508a69948cded53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7512b498705c4b26b508a69948cded53" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41837 } } }
I20260811 07:39:07.377305 24574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:07.377207 24572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7512b498705c4b26b508a69948cded53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7512b498705c4b26b508a69948cded53" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41837 } } }
I20260811 07:39:07.377645 24572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:07.377795 24583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:07.381071 24583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:07.386745 24583 catalog_manager.cc:1347] Generated new cluster ID: 978dbdc19af2473ab7d0a23cc9e4085f
I20260811 07:39:07.386807 24583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:07.414573 24583 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:07.415853 24583 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:07.430883 24583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7512b498705c4b26b508a69948cded53: Generated new TSK 0
I20260811 07:39:07.431733 24583 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:39:07.468524 24569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:07.445088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51608 (local address 127.18.72.254:41837)
0811 07:39:07.445230 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:07.445234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:07.445421 (+ 187us) server_negotiation.cc:407] Connection header received
0811 07:39:07.445595 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:07.445598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:07.445648 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:07.445733 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0811 07:39:07.445983 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.446510 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.447714 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.448000 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.448076 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.448423 (+ 347us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 07:39:07.468076 (+ 19653us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37029/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 07:39:07.468372 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51608: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37029/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 07:39:07.468613 24569 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51608: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37029/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 07:39:07.468732 24598 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:07.445013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41837 (local address 127.0.0.1:51608)
0811 07:39:07.445325 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:07.445340 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:07.445459 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:07.445756 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:07.445759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:07.445767 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 07:39:07.445879 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:07.445886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.446613 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.446616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:07.447593 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:07.447603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.448114 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.448118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:07.448216 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.448251 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 07:39:07.468296 (+ 20045us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37029/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 07:39:07.468515 (+ 219us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.72.254:41837: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37029/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":113,"threads_started":1}
E20260811 07:39:07.469264 24599 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 07:39:07.469332 24599 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 07:39:07.469445 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 24491
2026-08-11T07:39:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 07:39:07.515202 18723 mini_oidc.cc:152] Starting JWKS server
I20260811 07:39:07.516142 18723 webserver.cc:460] Webserver started at https://127.0.0.1:33155/ using document root <none> and password file <none>
I20260811 07:39:07.516263 18723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 07:39:07.516548 18723 webserver.cc:460] Webserver started at http://127.0.0.1:43755/ using document root <none> and password file <none>
2026-08-11T07:39:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:07Z Disabled control of system clock
I20260811 07:39:07.526707 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41031
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:41031
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33155/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 07:39:07.639256 24607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:07.639514 24607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:07.639552 24607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:07.643503 24607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:07.643556 24607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:07.643572 24607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:07.643589 24607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:07.643604 24607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 07:39:07.643669 24607 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 07:39:07.643689 24607 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33155/jwks.json
I20260811 07:39:07.648406 24607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:41031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33155/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:07.649447 24607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:07.650846 24607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:07.659370 24612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:07.659376 24615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:07.659433 24607 server_base.cc:1034] running on GCE node
W20260811 07:39:07.659375 24613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:07.660219 24607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:07.661309 24607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:07.662474 24607 hybrid_clock.cc:648] HybridClock initialized: now 1786433947662454 us; error 32 us; skew 500 ppm
I20260811 07:39:07.664463 24607 webserver.cc:460] Webserver started at http://127.18.72.254:40387/ using document root <none> and password file <none>
I20260811 07:39:07.665056 24607 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:07.665120 24607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:07.665383 24607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:07.667438 24607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "a994230e1aa948019a108d672af55282"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "86fd7b536889f5a30e82ff53d27caba0"
server_key_iv: "a98c7e574adbb0d24b9dd1828c4b9976"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.668076 24607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "a994230e1aa948019a108d672af55282"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "86fd7b536889f5a30e82ff53d27caba0"
server_key_iv: "a98c7e574adbb0d24b9dd1828c4b9976"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.671722 24607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:07.674177 24621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:07.675411 24607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:07.675545 24607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "a994230e1aa948019a108d672af55282"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "86fd7b536889f5a30e82ff53d27caba0"
server_key_iv: "a98c7e574adbb0d24b9dd1828c4b9976"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.675645 24607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:07.692343 24607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:07.693004 24607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:07.693197 24607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:07.699785 24607 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:41031
I20260811 07:39:07.699843 24680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:41031 every 8 connection(s)
I20260811 07:39:07.700814 24607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:07.702495 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 24607
I20260811 07:39:07.702654 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:07.702940 18723 external_mini_cluster.cc:1428] Setting key acd7517942a3df8924a8d579f856818a
I20260811 07:39:07.703918 24681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:07.709576 24681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282: Bootstrap starting.
I20260811 07:39:07.711853 24681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:07.711997 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:07.704377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.18.72.254:41031)
0811 07:39:07.705018 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:07.705033 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:07.705067 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:39:07.705708 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:07.705724 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:07.706093 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:07.706441 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:07.706869 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.707807 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.708717 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.709204 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.709400 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.710288 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:07.710293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:07.710305 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:07.710677 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:07.710730 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:07.710749 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:07.710858 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:07.710981 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 07:39:07.711245 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":143,"threads_started":1}
I20260811 07:39:07.712800 24681 log.cc:824] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:07.714720 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:39:07.714931 24681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282: No bootstrap required, opened a new log
I20260811 07:39:07.718495 24681 raft_consensus.cc:348] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a994230e1aa948019a108d672af55282" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41031 } }
I20260811 07:39:07.718699 24681 raft_consensus.cc:374] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:07.718779 24681 raft_consensus.cc:729] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a994230e1aa948019a108d672af55282, State: Initialized, Role: FOLLOWER
I20260811 07:39:07.719233 24681 consensus_queue.cc:260] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a994230e1aa948019a108d672af55282" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41031 } }
I20260811 07:39:07.719384 24681 raft_consensus.cc:388] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:07.719473 24681 raft_consensus.cc:482] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:07.719573 24681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:07.720490 24681 raft_consensus.cc:504] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a994230e1aa948019a108d672af55282" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41031 } }
I20260811 07:39:07.720817 24681 leader_election.cc:302] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a994230e1aa948019a108d672af55282; no voters:
I20260811 07:39:07.721125 24681 leader_election.cc:288] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:07.721259 24687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:07.721518 24687 raft_consensus.cc:686] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [term 1 LEADER]: Becoming Leader. State: Replica: a994230e1aa948019a108d672af55282, State: Running, Role: LEADER
I20260811 07:39:07.721814 24687 consensus_queue.cc:237] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a994230e1aa948019a108d672af55282" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41031 } }
I20260811 07:39:07.722396 24681 sys_catalog.cc:564] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:07.723538 24689 sys_catalog.cc:455] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a994230e1aa948019a108d672af55282. Latest consensus state: current_term: 1 leader_uuid: "a994230e1aa948019a108d672af55282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a994230e1aa948019a108d672af55282" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41031 } } }
I20260811 07:39:07.723673 24689 sys_catalog.cc:458] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:07.723985 24688 sys_catalog.cc:455] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a994230e1aa948019a108d672af55282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a994230e1aa948019a108d672af55282" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41031 } } }
I20260811 07:39:07.724099 24688 sys_catalog.cc:458] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:07.724172 24696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:07.727171 24696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:07.732892 24696 catalog_manager.cc:1347] Generated new cluster ID: ee5bed6d856746bcb052ee150fa47e36
I20260811 07:39:07.732956 24696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:07.758854 24696 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:07.759816 24696 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:07.769690 24696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a994230e1aa948019a108d672af55282: Generated new TSK 0
I20260811 07:39:07.770498 24696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:39:07.790637 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:07.785509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.18.72.254:41031)
0811 07:39:07.785655 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:07.785660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:07.785820 (+ 160us) server_negotiation.cc:407] Connection header received
0811 07:39:07.786006 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:07.786008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:07.786055 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:07.786140 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0811 07:39:07.786419 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.786932 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.788236 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.788462 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.788545 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.788883 (+ 338us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 07:39:07.790264 (+ 1381us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 07:39:07.790493 (+ 229us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36230: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 07:39:07.790712 24685 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36230: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 07:39:07.792610 24712 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:07.785424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41031 (local address 127.0.0.1:36230)
0811 07:39:07.785726 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:07.785742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:07.785867 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:07.786173 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:07.786176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:07.786185 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 07:39:07.786313 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:07.786320 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.787071 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.787075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:07.788112 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:07.788123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:07.788583 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:07.788587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:07.788676 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:07.788705 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 07:39:07.790428 (+ 1723us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 07:39:07.792439 (+ 2011us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.72.254:41031: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33155/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":1813,"thread_start_us":106,"threads_started":1}
I20260811 07:39:07.793538 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 24607
E20260811 07:39:07.801874 24713 webserver.cc:533] Webserver: sslize failed:
E20260811 07:39:07.801963 24713 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T07:39:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (320 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T07:39:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:07Z Disabled control of system clock
I20260811 07:39:07.845006 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:43197
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:32773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:43197
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/testchainca.cert with env {}
W20260811 07:39:07.956626 24717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:07.956925 24717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:07.956998 24717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:07.960958 24717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:07.961048 24717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:07.961079 24717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:07.961117 24717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:07.961149 24717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 07:39:07.961184 24717 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 07:39:07.961211 24717 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 07:39:07.966615 24717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:32773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:43197
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:43197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:07.967696 24717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:07.968765 24717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:07.977339 24723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:07.977406 24717 server_base.cc:1034] running on GCE node
W20260811 07:39:07.977380 24722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:07.977346 24725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:07.978209 24717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:07.979265 24717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:07.980439 24717 hybrid_clock.cc:648] HybridClock initialized: now 1786433947980416 us; error 30 us; skew 500 ppm
I20260811 07:39:07.982384 24717 webserver.cc:460] Webserver started at http://127.18.72.254:36225/ using document root <none> and password file <none>
I20260811 07:39:07.982991 24717 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:07.983055 24717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:07.983315 24717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:07.985441 24717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "60d5431420a840de8e75d72c4f676e36"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "e1380c11f5cd925357aab13a6e51d8bc"
server_key_iv: "979ca9bce44d3e8efc8f50fbd8d20b53"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.986025 24717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "60d5431420a840de8e75d72c4f676e36"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "e1380c11f5cd925357aab13a6e51d8bc"
server_key_iv: "979ca9bce44d3e8efc8f50fbd8d20b53"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.989796 24717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:39:07.992314 24733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:07.993589 24717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:07.993754 24717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "60d5431420a840de8e75d72c4f676e36"
format_stamp: "Formatted at 2026-08-11 07:39:07 on dist-test-slave-zkpd"
server_key: "e1380c11f5cd925357aab13a6e51d8bc"
server_key_iv: "979ca9bce44d3e8efc8f50fbd8d20b53"
server_key_version: "encryptionkey@0"
I20260811 07:39:07.993853 24717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:08.012318 24717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:08.013042 24717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:08.013243 24717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:08.020170 24717 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:43197
I20260811 07:39:08.020179 24790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:43197 every 8 connection(s)
I20260811 07:39:08.021221 24717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:08.024509 24792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:08.029992 24792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36: Bootstrap starting.
I20260811 07:39:08.030689 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 24717
I20260811 07:39:08.030818 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:08.031131 18723 external_mini_cluster.cc:1428] Setting key cb12263bdfe7b8797d809b10447bf296
I20260811 07:39:08.032989 24792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:08.033758 24792 log.cc:824] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:08.035866 24792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36: No bootstrap required, opened a new log
I20260811 07:39:08.040287 24792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5431420a840de8e75d72c4f676e36" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43197 } }
I20260811 07:39:08.040551 24792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:08.040637 24792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d5431420a840de8e75d72c4f676e36, State: Initialized, Role: FOLLOWER
I20260811 07:39:08.041059 24795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:08.032493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.18.72.254:43197)
0811 07:39:08.033178 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:08.033198 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:08.033239 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:39:08.033897 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:08.033923 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:08.034443 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:08.034804 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:08.035145 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:08.036305 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:08.037523 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:08.037988 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:08.038118 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:08.039030 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:08.039035 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:08.039048 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:08.039430 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:08.039484 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:08.039513 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:08.039592 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:08.039781 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 07:39:08.040115 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":151,"threads_started":1}
I20260811 07:39:08.041086 24792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5431420a840de8e75d72c4f676e36" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43197 } }
I20260811 07:39:08.041277 24792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:08.041347 24792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:08.041435 24792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:08.042460 24792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5431420a840de8e75d72c4f676e36" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43197 } }
I20260811 07:39:08.042789 24792 leader_election.cc:302] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60d5431420a840de8e75d72c4f676e36; no voters:
I20260811 07:39:08.043152 24792 leader_election.cc:288] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:08.043265 24797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:08.043524 24797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [term 1 LEADER]: Becoming Leader. State: Replica: 60d5431420a840de8e75d72c4f676e36, State: Running, Role: LEADER
I20260811 07:39:08.044042 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:39:08.044219 24797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5431420a840de8e75d72c4f676e36" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43197 } }
I20260811 07:39:08.044546 24792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:08.046095 24799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60d5431420a840de8e75d72c4f676e36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5431420a840de8e75d72c4f676e36" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43197 } } }
I20260811 07:39:08.046226 24799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:08.046595 24798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60d5431420a840de8e75d72c4f676e36. Latest consensus state: current_term: 1 leader_uuid: "60d5431420a840de8e75d72c4f676e36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5431420a840de8e75d72c4f676e36" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43197 } } }
I20260811 07:39:08.046693 24798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:08.046793 24809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:08.049856 24809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:08.055280 24809 catalog_manager.cc:1347] Generated new cluster ID: cb9c17f6cc9e41b1b180e4e9c62b8861
I20260811 07:39:08.055374 24809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:08.068641 24809 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:08.069896 24809 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:08.082253 24809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60d5431420a840de8e75d72c4f676e36: Generated new TSK 0
I20260811 07:39:08.083140 24809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:39:08.192723 24795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:08.093880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33430 (local address 127.18.72.254:43197)
0811 07:39:08.094014 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:08.094019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:08.094172 (+ 153us) server_negotiation.cc:407] Connection header received
0811 07:39:08.094324 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:08.094326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:08.094376 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:08.094470 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0811 07:39:08.094770 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:08.095325 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:08.096615 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:08.096840 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:08.096922 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:08.097242 (+ 320us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 07:39:08.192333 (+ 95091us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 07:39:08.192603 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33430: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 07:39:08.192826 24795 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33430: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 07:39:08.192974 24823 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:08.093744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:43197 (local address 127.0.0.1:33430)
0811 07:39:08.094052 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:08.094068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:08.094191 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:08.094501 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:08.094504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:08.094512 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 07:39:08.094632 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:08.094640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:08.095449 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:08.095453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:08.096480 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:08.096491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:08.096959 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:08.096963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:08.097049 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:08.097076 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 07:39:08.192575 (+ 95499us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 07:39:08.192779 (+ 204us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.72.254:43197: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":101,"threads_started":1}
I20260811 07:39:08.193681 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 24717
2026-08-11T07:39:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (381 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (131 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:08 dist-test-slave-zkpd krb5kdc[24836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:08 dist-test-slave-zkpd krb5kdc[24836](info): set up 2 sockets
Aug 11 07:39:08 dist-test-slave-zkpd krb5kdc[24836](info): commencing operation
krb5kdc: starting...
W20260811 07:39:09.533428 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:09 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:39:09.658896 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:35947
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:35947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:09.771703 24850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:09.772066 24850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:09.772161 24850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:09.776106 24850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:09.776192 24850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:09.776233 24850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:09.776269 24850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:09.776293 24850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:09.781534 24850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:35947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:35947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:09.782671 24850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:09.783792 24850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:09.792305 24857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:09.792446 24856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:09.792317 24859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:09.792351 24850 server_base.cc:1034] running on GCE node
I20260811 07:39:09.793089 24850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:09.794188 24850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:09.795361 24850 hybrid_clock.cc:648] HybridClock initialized: now 1786433949795335 us; error 42 us; skew 500 ppm
Aug 11 07:39:09 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433949, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:09.801996 24850 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:39:09.802882 24850 webserver.cc:460] Webserver started at http://127.18.72.254:37215/ using document root <none> and password file <none>
I20260811 07:39:09.803452 24850 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:09.803545 24850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:09.803784 24850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:09.806028 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "444babef9e764f60b69c143d580a908d"
format_stamp: "Formatted at 2026-08-11 07:39:09 on dist-test-slave-zkpd"
server_key: "bbede58be8805f93c2d4ef0bcdcc80ea"
server_key_iv: "b77a3c5fbc863a74bd796323ac5b580d"
server_key_version: "encryptionkey@0"
I20260811 07:39:09.806591 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "444babef9e764f60b69c143d580a908d"
format_stamp: "Formatted at 2026-08-11 07:39:09 on dist-test-slave-zkpd"
server_key: "bbede58be8805f93c2d4ef0bcdcc80ea"
server_key_iv: "b77a3c5fbc863a74bd796323ac5b580d"
server_key_version: "encryptionkey@0"
I20260811 07:39:09.810400 24850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 07:39:09.812824 24868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:09.814057 24850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 07:39:09.814205 24850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "444babef9e764f60b69c143d580a908d"
format_stamp: "Formatted at 2026-08-11 07:39:09 on dist-test-slave-zkpd"
server_key: "bbede58be8805f93c2d4ef0bcdcc80ea"
server_key_iv: "b77a3c5fbc863a74bd796323ac5b580d"
server_key_version: "encryptionkey@0"
I20260811 07:39:09.814313 24850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:09.847790 24850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:09.850934 24850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:09.851152 24850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:09.858712 24850 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:35947
I20260811 07:39:09.858789 24924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:35947 every 8 connection(s)
I20260811 07:39:09.859748 24850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:09.862815 24925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:09.866051 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 24850
I20260811 07:39:09.866159 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:09.866477 18723 external_mini_cluster.cc:1428] Setting key 91c7cfa1c2aa75b9e8fec521e7e6aac0
I20260811 07:39:09.869565 24925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d: Bootstrap starting.
I20260811 07:39:09.872408 24925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:09.873210 24925 log.cc:824] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:09.875483 24925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d: No bootstrap required, opened a new log
Aug 11 07:39:09 dist-test-slave-zkpd krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:09.879165 24925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444babef9e764f60b69c143d580a908d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35947 } }
I20260811 07:39:09.879402 24925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:09.879462 24925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444babef9e764f60b69c143d580a908d, State: Initialized, Role: FOLLOWER
I20260811 07:39:09.880014 24925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444babef9e764f60b69c143d580a908d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35947 } }
I20260811 07:39:09.880214 24925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:09.880322 24925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:09.880455 24925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:09.881453 24925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444babef9e764f60b69c143d580a908d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35947 } }
I20260811 07:39:09.881819 24925 leader_election.cc:302] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 444babef9e764f60b69c143d580a908d; no voters:
I20260811 07:39:09.882215 24925 leader_election.cc:288] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:09.882290 24931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:09.882488 24931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [term 1 LEADER]: Becoming Leader. State: Replica: 444babef9e764f60b69c143d580a908d, State: Running, Role: LEADER
I20260811 07:39:09.882923 24931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444babef9e764f60b69c143d580a908d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35947 } }
I20260811 07:39:09.883391 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:09.868064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.18.72.254:35947)
0811 07:39:09.868530 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:09.868551 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:09.868596 (+ 45us) server_negotiation.cc:407] Connection header received
0811 07:39:09.869413 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:09.869439 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:09.869886 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:09.870249 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:09.870964 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:09.872452 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:09.873372 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:09.874073 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:09.874257 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:09.877240 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:09.877245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:09.877267 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:09.877295 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:09.879687 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:09.880312 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:09.880318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:09.880325 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:09.880499 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:09.880840 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:09.880844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:09.880845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:09.881277 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:09.881498 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:09.882099 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 07:39:09.882465 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":126,"threads_started":1}
I20260811 07:39:09.883711 24925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:09.884390 24931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "444babef9e764f60b69c143d580a908d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444babef9e764f60b69c143d580a908d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35947 } } }
I20260811 07:39:09.884593 24931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:09.884852 24932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 444babef9e764f60b69c143d580a908d. Latest consensus state: current_term: 1 leader_uuid: "444babef9e764f60b69c143d580a908d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444babef9e764f60b69c143d580a908d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35947 } } }
I20260811 07:39:09.884966 24932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:09.885684 24936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:09.888993 24936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:09.896345 24936 catalog_manager.cc:1347] Generated new cluster ID: cc85583995c5426688f719226f7cee7e
I20260811 07:39:09.896427 24936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:09.904049 24936 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:09.905272 24936 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:09.915220 24936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 444babef9e764f60b69c143d580a908d: Generated new TSK 0
I20260811 07:39:09.916114 24936 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:39:09.950512 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35947
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:10.066284 24954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:10.066540 24954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:10.066582 24954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:10.070734 24954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:10.070798 24954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:10.070888 24954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:10.076176 24954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:10.077306 24954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:10.078373 24954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:10.088459 24960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:10.088588 24959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:10.088487 24962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:10.088527 24954 server_base.cc:1034] running on GCE node
I20260811 07:39:10.089040 24954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:10.089660 24954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:10.090819 24954 hybrid_clock.cc:648] HybridClock initialized: now 1786433950090806 us; error 23 us; skew 500 ppm
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.093601 24954 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:39:10.094429 24954 webserver.cc:460] Webserver started at http://127.18.72.193:33911/ using document root <none> and password file <none>
I20260811 07:39:10.094952 24954 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:10.095010 24954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:10.095173 24954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:10.097289 24954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "2e28007d7c9a4e7f857343c2544efea2"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "02d1789505b0b59f90d65fadd5b63cef"
server_key_iv: "3fa504f98747f3066d997a62fa71d748"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.097803 24954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "2e28007d7c9a4e7f857343c2544efea2"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "02d1789505b0b59f90d65fadd5b63cef"
server_key_iv: "3fa504f98747f3066d997a62fa71d748"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.101362 24954 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:39:10.104120 24969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.105305 24954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:10.105430 24954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "2e28007d7c9a4e7f857343c2544efea2"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "02d1789505b0b59f90d65fadd5b63cef"
server_key_iv: "3fa504f98747f3066d997a62fa71d748"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.105538 24954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:10.126727 24954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:10.130522 24954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:10.130741 24954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:10.131364 24954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:10.132282 24954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:10.132360 24954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.132452 24954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:10.132501 24954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.141592 24954 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:40287
I20260811 07:39:10.141665 25090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:40287 every 8 connection(s)
I20260811 07:39:10.142515 24954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:10.148113 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 24954
I20260811 07:39:10.148245 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:10.148524 18723 external_mini_cluster.cc:1428] Setting key 28fb52bf2f9a9fb5bafc7587ff9c16c5
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:10.157699 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.144366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:45613 (local address 127.18.72.254:35947)
0811 07:39:10.144504 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.144508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.145366 (+ 858us) server_negotiation.cc:407] Connection header received
0811 07:39:10.146768 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.146771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.146845 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.146955 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.148214 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.148856 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.150343 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.150570 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.150653 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.153883 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:10.153886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:10.153888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:10.153911 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:10.155367 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.156042 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.156045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.156046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.156119 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.156538 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.156541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.156542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.156707 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:10.156779 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:10.157465 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 07:39:10.157578 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:39:10.158906 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.144752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:35947 (local address 127.18.72.193:45613)
0811 07:39:10.145221 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:10.145262 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:10.146550 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:10.147140 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:10.147147 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:10.147501 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:10.148060 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:10.148077 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.149011 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.149014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:10.150211 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:10.150223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.150730 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.150737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:10.150986 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.151603 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:10.151629 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:10.153673 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:10.155508 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:10.155517 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:10.155537 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:10.155945 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:10.156219 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:10.156224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:10.156227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:10.156432 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:10.156822 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:10.156831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:10.157191 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 07:39:10.157422 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 07:39:10.157801 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
I20260811 07:39:10.160233 25091 heartbeater.cc:347] Connected to a master server at 127.18.72.254:35947
I20260811 07:39:10.160492 25091 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:10.160997 25091 heartbeater.cc:511] Master 127.18.72.254:35947 requested a full tablet report, sending...
I20260811 07:39:10.162708 24886 ts_manager.cc:194] Registered new tserver with Master: 2e28007d7c9a4e7f857343c2544efea2 (127.18.72.193:40287)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
I20260811 07:39:10.164371 24886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:45613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:39:10.179347 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35947
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:10.294160 25096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:10.294452 25096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:10.294498 25096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:10.298601 25096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:10.298658 25096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:10.298758 25096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:10.304064 25096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:10.305202 25096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:10.306283 25096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:10.316105 25102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:10.316138 25103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:10.316286 25105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:10.316417 25096 server_base.cc:1034] running on GCE node
I20260811 07:39:10.316867 25096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:10.317505 25096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:10.318696 25096 hybrid_clock.cc:648] HybridClock initialized: now 1786433950318674 us; error 29 us; skew 500 ppm
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.321512 25096 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:39:10.322408 25096 webserver.cc:460] Webserver started at http://127.18.72.194:44451/ using document root <none> and password file <none>
I20260811 07:39:10.323004 25096 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:10.323071 25096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:10.323293 25096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:10.325462 25096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "ab7ecdac84984653af5f13914f56909a"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "97680370c5b2d7a224577aa9366d6ffd"
server_key_iv: "8997492b1af40e99a2af0a6a75510c2b"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.326045 25096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "ab7ecdac84984653af5f13914f56909a"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "97680370c5b2d7a224577aa9366d6ffd"
server_key_iv: "8997492b1af40e99a2af0a6a75510c2b"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.329855 25096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:39:10.332342 25113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.333542 25096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:10.333663 25096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "ab7ecdac84984653af5f13914f56909a"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "97680370c5b2d7a224577aa9366d6ffd"
server_key_iv: "8997492b1af40e99a2af0a6a75510c2b"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.333776 25096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:10.356791 25096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:10.359884 25096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:10.360080 25096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:10.360627 25096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:10.361440 25096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:10.361487 25096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.361528 25096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:10.361543 25096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.370625 25096 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:34457
I20260811 07:39:10.370715 25239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:34457 every 8 connection(s)
I20260811 07:39:10.371598 25096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:10.376747 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 25096
I20260811 07:39:10.376890 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:10.377177 18723 external_mini_cluster.cc:1428] Setting key bd42295aef98fd880e7d50831c4745d7
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:10.386859 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.373535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:59759 (local address 127.18.72.254:35947)
0811 07:39:10.373717 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.373722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.374636 (+ 914us) server_negotiation.cc:407] Connection header received
0811 07:39:10.376084 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.376087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.376143 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.376230 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.377594 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.378374 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.379596 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.380019 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.380143 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.382913 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:10.382916 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:10.382918 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:10.382941 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:10.384514 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.385189 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.385193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.385195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.385277 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.385707 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.385709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.385710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.385890 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:10.385992 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:10.386636 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 07:39:10.386739 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 07:39:10.388123 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.373950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:35947 (local address 127.18.72.194:59759)
0811 07:39:10.374467 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:10.374521 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:10.375807 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:10.376401 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:10.376407 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:10.376775 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:10.377412 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:10.377428 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.378524 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.378528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:10.379461 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:10.379471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.380161 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.380164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:10.380307 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.380919 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:10.380935 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:10.382734 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:10.384628 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:10.384637 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:10.384656 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:10.385064 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:10.385374 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:10.385379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:10.385383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:10.385609 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:10.386024 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:10.386032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:10.386359 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 07:39:10.386588 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 07:39:10.386966 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":159,"threads_started":1}
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:39:10.389355 25240 heartbeater.cc:347] Connected to a master server at 127.18.72.254:35947
I20260811 07:39:10.389626 25240 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:10.390156 25240 heartbeater.cc:511] Master 127.18.72.254:35947 requested a full tablet report, sending...
I20260811 07:39:10.391219 24886 ts_manager.cc:194] Registered new tserver with Master: ab7ecdac84984653af5f13914f56909a (127.18.72.194:34457)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
I20260811 07:39:10.391990 24886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:59759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:39:10.408453 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35947
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:10.523463 25246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:10.523746 25246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:10.523795 25246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:10.527873 25246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:10.527989 25246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:10.528082 25246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:10.533372 25246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:10.534584 25246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:10.535648 25246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:10.545459 25251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:10.545560 25246 server_base.cc:1034] running on GCE node
W20260811 07:39:10.545488 25252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:10.545500 25254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:10.546176 25246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:10.547039 25246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:10.548369 25246 hybrid_clock.cc:648] HybridClock initialized: now 1786433950548353 us; error 26 us; skew 500 ppm
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.551093 25246 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:39:10.551966 25246 webserver.cc:460] Webserver started at http://127.18.72.195:33985/ using document root <none> and password file <none>
I20260811 07:39:10.552510 25246 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:10.552569 25246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:10.552734 25246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:10.554750 25246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "86564435064942a890562bfa48698389"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "4920940da35b619650ef5ad5ae5dd9a1"
server_key_iv: "8245ccc1e3d075ef2043a53f4be0dbc2"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.555267 25246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "86564435064942a890562bfa48698389"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "4920940da35b619650ef5ad5ae5dd9a1"
server_key_iv: "8245ccc1e3d075ef2043a53f4be0dbc2"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.558938 25246 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:39:10.561391 25261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.562803 25246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:10.562987 25246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "86564435064942a890562bfa48698389"
format_stamp: "Formatted at 2026-08-11 07:39:10 on dist-test-slave-zkpd"
server_key: "4920940da35b619650ef5ad5ae5dd9a1"
server_key_iv: "8245ccc1e3d075ef2043a53f4be0dbc2"
server_key_version: "encryptionkey@0"
I20260811 07:39:10.563089 25246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:10.584134 25246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:10.587548 25246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:10.587774 25246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:10.588445 25246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:10.589316 25246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:10.589365 25246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.589439 25246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:10.589473 25246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:10.598497 25246 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:35849
I20260811 07:39:10.598580 25381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:35849 every 8 connection(s)
I20260811 07:39:10.599478 25246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:10.605765 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 25246
I20260811 07:39:10.605928 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:10.606220 18723 external_mini_cluster.cc:1428] Setting key 630abe2789714bbc7ac570ff8477f38b
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:10.614269 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.601464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:53755 (local address 127.18.72.254:35947)
0811 07:39:10.601640 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.601644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.602505 (+ 861us) server_negotiation.cc:407] Connection header received
0811 07:39:10.604060 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.604063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.604123 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.604237 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.605444 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.606065 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.607594 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.607884 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.608026 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.610772 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:10.610775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:10.610778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:10.610827 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:10.612379 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.612863 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.612866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.612867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.612942 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.613239 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.613242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.613243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.613420 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:10.613495 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:10.614030 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 07:39:10.614137 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:39:10.615059 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.601823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:35947 (local address 127.18.72.195:53755)
0811 07:39:10.602348 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:10.602393 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:10.603779 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:10.604424 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:10.604432 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:10.604794 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:10.605294 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:10.605307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.606199 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.606203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:10.607468 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:10.607480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.608088 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.608092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:10.608296 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.608994 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:10.609012 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:10.610605 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:10.612480 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:10.612484 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:10.612498 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:10.612748 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:10.613026 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:10.613029 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:10.613030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:10.613161 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:10.613509 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:10.613514 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:10.613758 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 07:39:10.613936 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 07:39:10.614185 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":130,"threads_started":1}
I20260811 07:39:10.616264 25382 heartbeater.cc:347] Connected to a master server at 127.18.72.254:35947
I20260811 07:39:10.616528 25382 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:10.617034 25382 heartbeater.cc:511] Master 127.18.72.254:35947 requested a full tablet report, sending...
I20260811 07:39:10.618141 24886 ts_manager.cc:194] Registered new tserver with Master: 86564435064942a890562bfa48698389 (127.18.72.195:35849)
I20260811 07:39:10.618755 24886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:53755
I20260811 07:39:10.620540 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/test-admin.keytab.
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.637935 18723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 07:39:10.640942 25389 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:10.639416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.18.72.193:40287)
0811 07:39:10.639705 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.639725 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.639752 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:39:10.639819 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.639824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.640084 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.640245 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.640748 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34654: BlockingRecv error: recv got EOF from 127.0.0.1:34654 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
W20260811 07:39:10.641023 18723 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5cc' of type FILE
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.642050 18723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.642964 18723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:39:10.652350 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.643840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.18.72.193:40287)
0811 07:39:10.644006 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.644009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.644023 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:10.644124 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.644126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.644163 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.644229 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.644712 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.645465 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.646249 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.646603 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.646684 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.648933 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:10.648942 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:10.648956 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:10.648980 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:10.650722 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.651104 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.651108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.651110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.651185 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.651405 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.651407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.651408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.651633 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:10.651776 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:10.652028 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 07:39:10.652217 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 07:39:10.656139 25389 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:10.655221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34670 (local address 127.18.72.193:40287)
0811 07:39:10.655357 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.655360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.655372 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:10.655485 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.655487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.655520 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.655583 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.656014 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34670: BlockingRecv error: recv got EOF from 127.0.0.1:34670 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 07:39:10.656503 18723 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5cc)
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.657431 18723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.658272 18723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:39:10.666649 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.659001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34678 (local address 127.18.72.193:40287)
0811 07:39:10.659112 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.659116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.659306 (+ 190us) server_negotiation.cc:407] Connection header received
0811 07:39:10.659420 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.659422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.659458 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.659521 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.660011 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.660512 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.661276 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.661491 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.661565 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.663642 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:10.663645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:10.663647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:10.663668 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:10.665075 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.665463 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.665466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.665467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.665542 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.665795 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.665798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.665799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.665976 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:10.666081 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:10.666271 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 07:39:10.666387 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 07:39:10.669399 25389 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:10.668500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.18.72.193:40287)
0811 07:39:10.668630 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.668633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.668645 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:10.668747 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.668749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.668782 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.668850 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.669285 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34684: BlockingRecv error: recv got EOF from 127.0.0.1:34684 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 07:39:10.669464 18723 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786433920963264-18723-0/krb5kdc/krb5cc' of type FILE
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.670439 18723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:10.671280 18723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[24836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:39:10.679970 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:10.672044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.18.72.193:40287)
0811 07:39:10.672180 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:10.672185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:10.672315 (+ 130us) server_negotiation.cc:407] Connection header received
0811 07:39:10.672472 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:10.672475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:10.672514 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:10.672601 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:10.673101 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.673586 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.674384 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:10.674613 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:10.674718 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:10.676923 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:10.676927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:10.676930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:10.676953 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:10.678368 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.678781 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.678786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.678788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.678873 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:10.679151 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:10.679155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:10.679156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:10.679314 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:10.679445 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:10.679673 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 07:39:10.679781 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 07:39:10.681003 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 24954
I20260811 07:39:10.690426 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 25096
I20260811 07:39:10.701320 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 25246
I20260811 07:39:10.708207 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 24850
2026-08-11T07:39:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2253 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[25402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[25402](info): set up 2 sockets
Aug 11 07:39:10 dist-test-slave-zkpd krb5kdc[25402](info): commencing operation
krb5kdc: starting...
W20260811 07:39:11.785236 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:11 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.254.keytab.
Entry for principal oryx/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.254.keytab.
I20260811 07:39:11.914609 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45797
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.254.keytab
--principal=oryx/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:45797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:12.029644 25419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:12.029966 25419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:12.030047 25419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:12.034181 25419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:12.034271 25419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:12.034309 25419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:12.034366 25419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:12.034400 25419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:12.039685 25419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:45797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.254.keytab
--principal=oryx/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:12.040803 25419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:12.041841 25419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:12.050638 25425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:12.050624 25427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:12.050625 25424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:12.050820 25419 server_base.cc:1034] running on GCE node
I20260811 07:39:12.051445 25419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:12.052601 25419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:12.053761 25419 hybrid_clock.cc:648] HybridClock initialized: now 1786433952053743 us; error 61 us; skew 500 ppm
Aug 11 07:39:12 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:12.056598 25419 init.cc:377] Logged in from keytab as oryx/127.18.72.254@KRBTEST.COM (short username oryx)
I20260811 07:39:12.057421 25419 webserver.cc:460] Webserver started at http://127.18.72.254:42829/ using document root <none> and password file <none>
I20260811 07:39:12.057953 25419 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:12.058010 25419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:12.058175 25419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:12.060304 25419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "7461a49810784a64b6c0e21a9dd398e6"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "f5323ed14d0ead75a24784502839f741"
server_key_iv: "17fc04dbe5deec6ff0abfeda603c8de0"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.060839 25419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "7461a49810784a64b6c0e21a9dd398e6"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "f5323ed14d0ead75a24784502839f741"
server_key_iv: "17fc04dbe5deec6ff0abfeda603c8de0"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.064605 25419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:39:12.067129 25434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.068445 25419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 07:39:12.068573 25419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "7461a49810784a64b6c0e21a9dd398e6"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "f5323ed14d0ead75a24784502839f741"
server_key_iv: "17fc04dbe5deec6ff0abfeda603c8de0"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.068676 25419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:12.094682 25419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:12.097901 25419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:12.098104 25419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:12.105343 25419 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:45797
I20260811 07:39:12.105639 25486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:45797 every 8 connection(s)
I20260811 07:39:12.106400 25419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:12.109356 25487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:12.111975 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 25419
I20260811 07:39:12.112120 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:12.112455 18723 external_mini_cluster.cc:1428] Setting key df1814fb6724875f886dae7a0213dd6b
I20260811 07:39:12.116057 25487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: Bootstrap starting.
I20260811 07:39:12.118703 25487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:12.119553 25487 log.cc:824] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: Log is configured to *not* fsync() on all Append() calls
Aug 11 07:39:12 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:12.121912 25487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: No bootstrap required, opened a new log
I20260811 07:39:12.126384 25487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } }
I20260811 07:39:12.126679 25487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:12.126777 25487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7461a49810784a64b6c0e21a9dd398e6, State: Initialized, Role: FOLLOWER
I20260811 07:39:12.127326 25487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } }
I20260811 07:39:12.127488 25487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:12.127552 25487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:12.127648 25487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:12.128423 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:12.113925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.18.72.254:45797)
0811 07:39:12.114359 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:12.114380 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:12.114417 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:39:12.115077 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:12.115095 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:12.115442 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:12.115726 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:12.116690 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.117694 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.118884 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.119390 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.119523 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:12.122371 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:12.122377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:12.122395 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:12.122426 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:12.125104 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:12.125535 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:12.125541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:12.125548 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:12.125694 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:12.125969 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:12.125972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:12.125974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:12.126511 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:12.126726 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:12.127024 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 07:39:12.127340 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":127,"threads_started":1}
I20260811 07:39:12.128881 25487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } }
I20260811 07:39:12.129242 25487 leader_election.cc:302] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7461a49810784a64b6c0e21a9dd398e6; no voters:
I20260811 07:39:12.129576 25487 leader_election.cc:288] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:12.129700 25492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:12.129931 25492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 1 LEADER]: Becoming Leader. State: Replica: 7461a49810784a64b6c0e21a9dd398e6, State: Running, Role: LEADER
I20260811 07:39:12.130327 25492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } }
I20260811 07:39:12.131058 25487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:12.131655 25495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } }
I20260811 07:39:12.131783 25495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:12.132097 25494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7461a49810784a64b6c0e21a9dd398e6. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } }
I20260811 07:39:12.132236 25499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:12.132254 25494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:12.135502 25499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:12.144042 25499 catalog_manager.cc:1347] Generated new cluster ID: e73456e2e8d0444db1bd5ea14054a285
I20260811 07:39:12.144127 25499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:12.155143 25499 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:12.156167 25499 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:12.174506 25499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: Generated new TSK 0
I20260811 07:39:12.175298 25499 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab.
Entry for principal oryx/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab.
I20260811 07:39:12.212742 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45797
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab
--principal=oryx/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:12.350991 25517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:12.351271 25517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:12.351318 25517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:12.355939 25517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:12.356012 25517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:12.356137 25517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:12.362339 25517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab
--principal=oryx/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:12.363564 25517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:12.364758 25517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:12.376163 25524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:12.376163 25526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:12.376166 25523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:12.376264 25517 server_base.cc:1034] running on GCE node
I20260811 07:39:12.376911 25517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:12.377614 25517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:12.378806 25517 hybrid_clock.cc:648] HybridClock initialized: now 1786433952378775 us; error 53 us; skew 500 ppm
Aug 11 07:39:12 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:12.381752 25517 init.cc:377] Logged in from keytab as oryx/127.18.72.193@KRBTEST.COM (short username oryx)
I20260811 07:39:12.382751 25517 webserver.cc:460] Webserver started at http://127.18.72.193:41963/ using document root <none> and password file <none>
I20260811 07:39:12.383358 25517 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:12.383455 25517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:12.383699 25517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:12.386215 25517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "08870894ac9541618bd48a2fe3dae028"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "f71837484c02967791d8b80920dfecc1"
server_key_iv: "57bd38d57547cd0ff88b1d3cc25ecf2e"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.386950 25517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "08870894ac9541618bd48a2fe3dae028"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "f71837484c02967791d8b80920dfecc1"
server_key_iv: "57bd38d57547cd0ff88b1d3cc25ecf2e"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.391100 25517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:39:12.393728 25535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.394997 25517 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:12.395156 25517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "08870894ac9541618bd48a2fe3dae028"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "f71837484c02967791d8b80920dfecc1"
server_key_iv: "57bd38d57547cd0ff88b1d3cc25ecf2e"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.395287 25517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:12.413712 25517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:12.416940 25517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:12.417160 25517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:12.417800 25517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:12.418682 25517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:12.418758 25517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.418825 25517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:12.418870 25517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.428436 25517 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:33509
I20260811 07:39:12.428457 25662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:33509 every 8 connection(s)
I20260811 07:39:12.429399 25517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:12.431500 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 25517
I20260811 07:39:12.431653 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:12.432075 18723 external_mini_cluster.cc:1428] Setting key dd321d626628bc5dbbf292230af5c6eb
Aug 11 07:39:12 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.72.194@KRBTEST.COM; defaulting to no policy
I20260811 07:39:12.444612 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:12.431324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:36283 (local address 127.18.72.254:45797)
0811 07:39:12.431474 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:12.431478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:12.432719 (+ 1241us) server_negotiation.cc:407] Connection header received
0811 07:39:12.434524 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:12.434527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:12.434582 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:12.434662 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:12.435910 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.436466 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.437642 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.437899 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.438016 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:12.440956 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:12.440964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:12.440967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:12.440991 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:12.442514 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:12.443131 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:12.443134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:12.443135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:12.443212 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:12.443530 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:12.443532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:12.443533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:12.443755 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:12.443844 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:12.444373 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 07:39:12.444485 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.194@KRBTEST.COM" created.
I20260811 07:39:12.445588 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:12.431957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.193:36283)
0811 07:39:12.432540 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:12.432603 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:12.434292 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:12.434855 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:12.434863 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:12.435227 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:12.435732 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:12.435745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.436611 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.436614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:12.437507 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:12.437517 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.438059 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.438063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:12.438241 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:12.438984 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:12.439001 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:12.440774 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:12.442676 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:12.442683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:12.442701 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:12.443031 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:12.443311 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:12.443313 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:12.443315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:12.443450 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:12.443839 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:12.443844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:12.444144 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 07:39:12.444321 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 07:39:12.444598 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":163,"threads_started":1}
I20260811 07:39:12.447028 25663 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45797
I20260811 07:39:12.447319 25663 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:12.447855 25663 heartbeater.cc:511] Master 127.18.72.254:45797 requested a full tablet report, sending...
I20260811 07:39:12.449502 25451 ts_manager.cc:194] Registered new tserver with Master: 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509)
I20260811 07:39:12.450635 25451 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:36283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab.
Entry for principal oryx/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab.
I20260811 07:39:12.462910 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45797
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab
--principal=oryx/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:12.575443 25668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:12.575681 25668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:12.575722 25668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:12.579676 25668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:12.579732 25668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:12.579834 25668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:12.585005 25668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab
--principal=oryx/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:12.586055 25668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:12.587133 25668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:12.596516 25674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:12.596585 25676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:12.596524 25673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:12.596897 25668 server_base.cc:1034] running on GCE node
I20260811 07:39:12.597326 25668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:12.597941 25668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:12.599125 25668 hybrid_clock.cc:648] HybridClock initialized: now 1786433952599103 us; error 42 us; skew 500 ppm
Aug 11 07:39:12 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:12.601933 25668 init.cc:377] Logged in from keytab as oryx/127.18.72.194@KRBTEST.COM (short username oryx)
I20260811 07:39:12.602711 25668 webserver.cc:460] Webserver started at http://127.18.72.194:46191/ using document root <none> and password file <none>
I20260811 07:39:12.603261 25668 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:12.603327 25668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:12.603487 25668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:12.605527 25668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "d3386466d6de4fb29244bcf1adc944d6"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "544f9682b632e29cf9cdae1fd0d0c087"
server_key_iv: "0949a1bcc61577445b44a387fad6075d"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.606045 25668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "d3386466d6de4fb29244bcf1adc944d6"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "544f9682b632e29cf9cdae1fd0d0c087"
server_key_iv: "0949a1bcc61577445b44a387fad6075d"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.609723 25668 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 07:39:12.612739 25683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.613981 25668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:39:12.614077 25668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "d3386466d6de4fb29244bcf1adc944d6"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "544f9682b632e29cf9cdae1fd0d0c087"
server_key_iv: "0949a1bcc61577445b44a387fad6075d"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.614159 25668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:12.633044 25668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:12.635797 25668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:12.636019 25668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:12.636682 25668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:12.637487 25668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:12.637535 25668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.637574 25668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:12.637590 25668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.646381 25668 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:45417
I20260811 07:39:12.646409 25812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:45417 every 8 connection(s)
I20260811 07:39:12.647265 25668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:12.649883 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 25668
I20260811 07:39:12.650012 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:12.650310 18723 external_mini_cluster.cc:1428] Setting key 7e65bca89c18c8b6d3e78435fafaeaad
Aug 11 07:39:12 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:39:12.661456 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:12.649066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:42029 (local address 127.18.72.254:45797)
0811 07:39:12.649204 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:12.649207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:12.650161 (+ 954us) server_negotiation.cc:407] Connection header received
0811 07:39:12.651786 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:12.651790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:12.651844 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:12.652005 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:12.653246 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.653780 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.654918 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.655146 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.655261 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:12.657954 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:12.657958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:12.657960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:12.657983 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:12.659387 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:12.659931 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:12.659934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:12.659936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:12.660016 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:12.660380 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:12.660385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:12.660387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:12.660589 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:12.660687 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:12.661176 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 07:39:12.661294 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:39:12.662349 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:12.649433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.194:42029)
0811 07:39:12.649971 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:12.650031 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:12.651562 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:12.652217 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:12.652224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:12.652621 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:12.653104 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:12.653117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.653915 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.653919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:12.654782 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:12.654791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.655299 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.655304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:12.655480 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:12.656102 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:12.656118 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:12.657778 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:12.659520 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:12.659525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:12.659540 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:12.659797 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:12.660110 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:12.660114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:12.660116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:12.660273 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:12.660694 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:12.660703 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:12.660943 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 07:39:12.661132 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 07:39:12.661456 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":187,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.195@KRBTEST.COM" created.
I20260811 07:39:12.663635 25813 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45797
I20260811 07:39:12.663932 25813 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:12.664420 25813 heartbeater.cc:511] Master 127.18.72.254:45797 requested a full tablet report, sending...
I20260811 07:39:12.665488 25451 ts_manager.cc:194] Registered new tserver with Master: d3386466d6de4fb29244bcf1adc944d6 (127.18.72.194:45417)
I20260811 07:39:12.666059 25451 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:42029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab.
Entry for principal oryx/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab.
I20260811 07:39:12.679271 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45797
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab
--principal=oryx/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:12.797437 25819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:12.797675 25819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:12.797725 25819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:12.801780 25819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:12.801831 25819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:12.801883 25819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:12.806870 25819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab
--principal=oryx/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:12.807803 25819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:12.808830 25819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:12.818663 25825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:12.818670 25829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:12.818724 25819 server_base.cc:1034] running on GCE node
W20260811 07:39:12.818679 25826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:12.819355 25819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:12.820016 25819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:12.821202 25819 hybrid_clock.cc:648] HybridClock initialized: now 1786433952821182 us; error 39 us; skew 500 ppm
Aug 11 07:39:12 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:12.824188 25819 init.cc:377] Logged in from keytab as oryx/127.18.72.195@KRBTEST.COM (short username oryx)
I20260811 07:39:12.825111 25819 webserver.cc:460] Webserver started at http://127.18.72.195:35125/ using document root <none> and password file <none>
I20260811 07:39:12.825713 25819 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:12.825810 25819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:12.826153 25819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:12.829066 25819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "25c89f1b811e4b08a8e691bf1fbce46a"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "8fbc0258d091513aab9456b6cfc860db"
server_key_iv: "d19f0fe0d73c83384ac839f17d5fba29"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.829698 25819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "25c89f1b811e4b08a8e691bf1fbce46a"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "8fbc0258d091513aab9456b6cfc860db"
server_key_iv: "d19f0fe0d73c83384ac839f17d5fba29"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.834115 25819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:39:12.836607 25836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.837801 25819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:12.837951 25819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "25c89f1b811e4b08a8e691bf1fbce46a"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "8fbc0258d091513aab9456b6cfc860db"
server_key_iv: "d19f0fe0d73c83384ac839f17d5fba29"
server_key_version: "encryptionkey@0"
I20260811 07:39:12.838064 25819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:12.868574 25819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:12.872160 25819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:12.872397 25819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:12.873031 25819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:12.874017 25819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:12.874089 25819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.874155 25819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:12.874198 25819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:12.883199 25819 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:33449
I20260811 07:39:12.883225 25966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:33449 every 8 connection(s)
I20260811 07:39:12.884109 25819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:12.886323 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 25819
I20260811 07:39:12.886461 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:12.886737 18723 external_mini_cluster.cc:1428] Setting key a5962872fabb7b1081be7c9ce5e24af1
Aug 11 07:39:12 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:12.897842 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:12.885938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:53931 (local address 127.18.72.254:45797)
0811 07:39:12.886103 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:12.886109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:12.887041 (+ 932us) server_negotiation.cc:407] Connection header received
0811 07:39:12.888478 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:12.888482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:12.888533 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:12.888620 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:12.889792 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.890318 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.891403 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.891609 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.891684 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:12.893980 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:12.893983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:12.893985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:12.894010 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:12.895606 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:12.896256 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:12.896260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:12.896262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:12.896351 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:12.896700 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:12.896704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:12.896706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:12.896904 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:12.897011 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:12.897517 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 07:39:12.897636 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:39:12.898731 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:12.886327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.195:53931)
0811 07:39:12.886870 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:12.886925 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:12.888272 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:12.888837 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:12.888844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:12.889191 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:12.889670 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:12.889682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.890424 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.890427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:12.891301 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:12.891310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:12.891732 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:12.891734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:12.891873 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:12.892386 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:12.892401 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:12.893826 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:12.895749 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:12.895755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:12.895770 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:12.896160 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:12.896448 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:12.896452 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:12.896454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:12.896597 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:12.897011 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:12.897018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:12.897268 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 07:39:12.897460 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 07:39:12.897783 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":172,"threads_started":1}
I20260811 07:39:12.900100 25967 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45797
I20260811 07:39:12.900372 25967 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:12.900930 25967 heartbeater.cc:511] Master 127.18.72.254:45797 requested a full tablet report, sending...
I20260811 07:39:12.901908 25449 ts_manager.cc:194] Registered new tserver with Master: 25c89f1b811e4b08a8e691bf1fbce46a (127.18.72.195:33449)
I20260811 07:39:12.902451 25449 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:53931
I20260811 07:39:12.912474 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.72.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.253.keytab.
Entry for principal oryx/127.18.72.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.253.keytab.
I20260811 07:39:12.940541 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.253:46723
--webserver_interface=127.18.72.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.253.keytab
--principal=oryx/127.18.72.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:45797,127.18.72.253:46723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:13.053577 25972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:13.053928 25972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:13.054035 25972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:13.058053 25972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:13.058132 25972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:13.058169 25972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:13.058226 25972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:13.058264 25972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:13.063493 25972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:45797,127.18.72.253:46723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.253.keytab
--principal=oryx/127.18.72.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.253:46723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.72.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
W20260811 07:39:13.063745 25972 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.72.254:45797,127.18.72.253:46723'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 07:39:13.064694 25972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:13.065728 25972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:13.074286 25980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.074352 25977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.074302 25978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:13.074662 25972 server_base.cc:1034] running on GCE node
I20260811 07:39:13.075212 25972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:13.076401 25972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:13.077560 25972 hybrid_clock.cc:648] HybridClock initialized: now 1786433953077531 us; error 40 us; skew 500 ppm
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:13.080358 25972 init.cc:377] Logged in from keytab as oryx/127.18.72.253@KRBTEST.COM (short username oryx)
I20260811 07:39:13.081202 25972 webserver.cc:460] Webserver started at http://127.18.72.253:39263/ using document root <none> and password file <none>
I20260811 07:39:13.081769 25972 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:13.081873 25972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:13.082108 25972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:13.084209 25972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data/instance:
uuid: "6251da0ffb45473fb96b1fb0abd9d14b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "6c649e8a11b105cb2dcc8c763b4abf95"
server_key_iv: "15a8a2be2b04105d0673ce10bf8a6653"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.084779 25972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal/instance:
uuid: "6251da0ffb45473fb96b1fb0abd9d14b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "6c649e8a11b105cb2dcc8c763b4abf95"
server_key_iv: "15a8a2be2b04105d0673ce10bf8a6653"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.088454 25972 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:39:13.090764 25988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.092061 25972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:13.092192 25972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
uuid: "6251da0ffb45473fb96b1fb0abd9d14b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "6c649e8a11b105cb2dcc8c763b4abf95"
server_key_iv: "15a8a2be2b04105d0673ce10bf8a6653"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.092300 25972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:13.130990 25972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:13.134485 25972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:13.134696 25972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:13.142023 25972 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.253:46723
I20260811 07:39:13.142323 26049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.253:46723 every 8 connection(s)
I20260811 07:39:13.143064 25972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
I20260811 07:39:13.145963 26050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:13.147424 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 25972
I20260811 07:39:13.150372 26050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.72.253@KRBTEST.COM
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.253@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:13.162254 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.152123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36748 (local address 127.18.72.254:45797)
0811 07:39:13.152296 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.152301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.152586 (+ 285us) server_negotiation.cc:407] Connection header received
0811 07:39:13.152706 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.152709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.152783 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.152855 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.153608 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.154292 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.155571 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.156047 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.156148 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.158761 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.158767 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.158772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.158803 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.160399 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.160881 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.160885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.160887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.160976 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.161259 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.161263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.161265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.161488 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.161598 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.161960 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.162090 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 07:39:13.163064 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.152054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:36748)
0811 07:39:13.152448 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.152497 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.152612 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.152927 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.152935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.153363 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.153484 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.153490 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.154412 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.154415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.155399 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.155418 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.156219 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.156227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.156387 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.156975 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.157010 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.158614 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.160518 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.160527 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.160534 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.160763 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.161069 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.161073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.161075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.161164 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.161596 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.161603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.161745 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.161899 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.162156 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":150,"threads_started":1}
I20260811 07:39:13.163069 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.149018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60418 (local address 127.18.72.253:46723)
0811 07:39:13.149448 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.149463 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.149498 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:39:13.150094 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.150109 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.150429 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.150755 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.151590 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.153032 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.154042 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.154602 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.154754 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.157671 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.157677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.157691 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.157719 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.160009 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.160476 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.160482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.160489 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.160602 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.160898 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.160903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.160905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.161309 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.161550 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.161769 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.162224 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":297,"thread_start_us":132,"threads_started":1}
I20260811 07:39:13.164534 26050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.253@KRBTEST.COM for oryx/127.18.72.253@KRBTEST.COM
I20260811 07:39:13.172204 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.165045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.0.0.1:60420)
0811 07:39:13.165161 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.165178 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.165259 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.165546 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.165548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.165673 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.165760 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.165764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.166758 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.166761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.167275 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.167282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.167671 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.167674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.167746 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.168195 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.168206 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.169354 (+ 1148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.170961 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.170963 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.170965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.171137 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.171371 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.171374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.171375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.171442 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.171786 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.171788 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.171860 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.171993 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.172094 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 07:39:13.172364 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.165274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.18.72.253:46723)
0811 07:39:13.165386 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.165389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.165400 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:39:13.165426 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.165428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.165458 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.165517 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.165859 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.166669 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.167376 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.167584 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.167657 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.169471 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.169474 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.169475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.169496 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.170871 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.171216 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.171219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.171220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.171288 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.171513 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.171515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.171516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.171665 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.171785 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.172033 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.172176 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":41}
I20260811 07:39:13.175292 26050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Bootstrap starting.
I20260811 07:39:13.177688 26050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:13.178431 26050 log.cc:824] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:13.180622 26050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: No bootstrap required, opened a new log
I20260811 07:39:13.183512 26050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } }
I20260811 07:39:13.183715 26050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:13.183749 26050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6251da0ffb45473fb96b1fb0abd9d14b, State: Initialized, Role: FOLLOWER
I20260811 07:39:13.184209 26050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } }
I20260811 07:39:13.184633 26057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } } }
I20260811 07:39:13.184767 26057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:39:13.185367 26050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: configured and running, proceeding with master startup.
W20260811 07:39:13.189878 26069 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:39:13.189972 26069 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:39:13.192128 25972 master_runner.cc:428] Detected that this master 6251da0ffb45473fb96b1fb0abd9d14b is joining an existing cluster
I20260811 07:39:13.192198 25972 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.72.253:46723
I20260811 07:39:13.203279 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.194362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:36752)
0811 07:39:13.194736 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.194748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.194868 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.195254 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.195257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.195553 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.195717 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.195725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.196660 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.196666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.197311 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.197320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.197790 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.197794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.197883 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.198697 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.198707 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.199092 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.201498 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.201504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.201507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.201829 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.202167 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.202170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.202172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.202257 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.202708 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.202711 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.202810 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.202954 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.203087 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":21,"thread_start_us":101,"threads_started":1}
I20260811 07:39:13.203771 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.194470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36752 (local address 127.18.72.254:45797)
0811 07:39:13.194600 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.194605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.194827 (+ 222us) server_negotiation.cc:407] Connection header received
0811 07:39:13.194992 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.194995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.195039 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.195123 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.195841 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.196551 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.197435 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.197671 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.197782 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.199233 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.199238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.199242 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.199271 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.201369 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.201953 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.201958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.201959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.202050 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.202367 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.202370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.202372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.202600 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.202712 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.203019 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.203144 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:39:13.203832 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.194538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.18.72.253:46723)
0811 07:39:13.194690 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.194695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.195205 (+ 510us) server_negotiation.cc:407] Connection header received
0811 07:39:13.195301 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.195305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.195349 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.195430 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.196003 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.196773 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.197949 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.198217 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.198349 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.200407 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.200409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.200412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.200439 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.202073 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.202561 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.202564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.202566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.202646 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.202916 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.202921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.202922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.203115 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.203205 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.203568 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.203675 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 07:39:13.203878 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.194761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.0.0.1:60434)
0811 07:39:13.195112 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.195127 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.195225 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.195472 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.195475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.195711 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.195839 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.195847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.196910 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.196913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.197846 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.197857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.198348 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.198352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.198464 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.199796 (+ 1332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.199810 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.200269 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.202180 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.202184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.202186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.202458 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.202742 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.202745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.202747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.202826 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.203237 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.203240 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.203313 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.203490 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.203606 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":1,"thread_start_us":127,"threads_started":1}
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.254@KRBTEST.COM for oryx/127.18.72.253@KRBTEST.COM
I20260811 07:39:13.218360 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.208931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.18.72.253:46723)
0811 07:39:13.209045 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.209049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.209384 (+ 335us) server_negotiation.cc:407] Connection header received
0811 07:39:13.209565 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.209568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.209617 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.209698 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.210271 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.211008 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.211845 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.212204 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.212302 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.214585 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.214589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.214592 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.214616 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.216302 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.216862 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.216866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.216867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.216950 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.217305 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.217308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.217310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.217511 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.217606 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.218092 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.218200 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:39:13.218573 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.208891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.0.0.1:60448)
0811 07:39:13.209245 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.209307 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.209435 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.209738 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.209743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.210067 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.210164 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.210169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.211120 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.211122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.211736 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.211744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.212317 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.212320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.212399 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.212881 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.212892 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.214453 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.216424 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.216435 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.216441 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.216744 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.217091 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.217097 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.217102 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.217207 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.217626 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.217636 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.217860 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.218025 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.218306 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":104,"threads_started":1}
I20260811 07:39:13.220146 25449 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.72.253:46723: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b"
member_type: NON_VOTER
last_known_addr {
host: "127.18.72.253"
port: 46723
}
attrs {
promote: true
}
}
dest_uuid: "7461a49810784a64b6c0e21a9dd398e6"
cas_config_opid_index: -1
I20260811 07:39:13.220825 25449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } }
I20260811 07:39:13.222738 26082 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6251da0ffb45473fb96b1fb0abd9d14b (127.18.72.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } }
I20260811 07:39:13.223366 26021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:13.224576 26082 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.72.253:46723
I20260811 07:39:13.224772 26021 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Refusing update from remote peer 7461a49810784a64b6c0e21a9dd398e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 07:39:13.224974 25972 master.cc:537] Master@127.18.72.253:46723 shutting down...
I20260811 07:39:13.225804 26082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } }
I20260811 07:39:13.225917 26082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:13.226102 26082 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
WARNING: no policy specified for oryx/127.18.72.252@KRBTEST.COM; defaulting to no policy
W20260811 07:39:13.229022 25436 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 07:39:13.230044 25436 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 -> Peer 6251da0ffb45473fb96b1fb0abd9d14b (127.18.72.253:46723): Couldn't send request to peer 6251da0ffb45473fb96b1fb0abd9d14b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 07:39:13.230093 25972 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Raft consensus shutting down.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.252@KRBTEST.COM" created.
I20260811 07:39:13.230436 25972 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:13.230566 25972 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: stopping tablet replica
I20260811 07:39:13.246973 25972 master.cc:552] Master@127.18.72.253:46723 shutdown complete.
I20260811 07:39:13.247041 25972 master_runner.cc:305] Clearing existing system tablet
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.252.keytab.
Entry for principal oryx/127.18.72.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.252.keytab.
I20260811 07:39:13.247352 25972 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:13.247414 25972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:13.248385 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.252:35873
--webserver_interface=127.18.72.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.252.keytab
--principal=oryx/127.18.72.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:45797,127.18.72.253:46723,127.18.72.252:35873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:39:13.248909 25972 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:39:13.249867 26090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.250110 25972 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:13.250234 25972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:39:13.250268 25972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
uuid: "6251da0ffb45473fb96b1fb0abd9d14b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "6c649e8a11b105cb2dcc8c763b4abf95"
server_key_iv: "15a8a2be2b04105d0673ce10bf8a6653"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.252668 25972 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:13.255146 25972 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 07:39:13.255239 25972 log.cc:1196] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 07:39:13.255501 25972 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Deleting consensus metadata
I20260811 07:39:13.255836 25972 master_runner.cc:315] Copying system tablet from 127.18.72.254:45797
I20260811 07:39:13.257061 25972 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: tablet copy: Beginning tablet copy session from remote peer at address 127.18.72.254:45797
I20260811 07:39:13.266780 26099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.257710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:36754)
0811 07:39:13.258037 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.258053 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.258178 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.258525 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.258529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.258759 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.258898 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.258907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.260261 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.260264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.261175 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.261185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.261816 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.261819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.261918 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.262528 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.262543 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.262968 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.265106 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.265110 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.265112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.265382 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.265684 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.265687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.265689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.265784 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.266206 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.266209 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.266311 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.266422 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.266570 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":99,"threads_started":1}
I20260811 07:39:13.266784 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.257774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36754 (local address 127.18.72.254:45797)
0811 07:39:13.257904 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.257908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.258158 (+ 250us) server_negotiation.cc:407] Connection header received
0811 07:39:13.258286 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.258289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.258363 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.258472 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.259102 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.260119 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.261324 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.261684 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.261809 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.263125 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.263129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.263131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.263161 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.264987 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.265495 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.265498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.265501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.265590 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.265877 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.265880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.265882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.266099 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.266193 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.266494 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.266617 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:39:13.267740 25471 tablet_copy_service.cc:140] P 7461a49810784a64b6c0e21a9dd398e6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6251da0ffb45473fb96b1fb0abd9d14b ({username='oryx', principal='oryx/127.18.72.253@KRBTEST.COM'} at 127.0.0.1:36754)
I20260811 07:39:13.268028 25471 tablet_copy_service.cc:161] P 7461a49810784a64b6c0e21a9dd398e6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6251da0ffb45473fb96b1fb0abd9d14b at {username='oryx', principal='oryx/127.18.72.253@KRBTEST.COM'} at 127.0.0.1:36754: session id = 6251da0ffb45473fb96b1fb0abd9d14b-00000000000000000000000000000000
I20260811 07:39:13.269894 25471 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 07:39:13.271776 25972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 07:39:13.274385 25972 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: tablet copy: Starting download of 0 data blocks...
I20260811 07:39:13.274617 25972 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: tablet copy: Starting download of 1 WAL segments...
I20260811 07:39:13.276615 25972 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 07:39:13.278604 25471 tablet_copy_service.cc:342] P 7461a49810784a64b6c0e21a9dd398e6: Request end of tablet copy session 6251da0ffb45473fb96b1fb0abd9d14b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.72.253@KRBTEST.COM'} at 127.0.0.1:36754
I20260811 07:39:13.278741 25471 tablet_copy_service.cc:433] P 7461a49810784a64b6c0e21a9dd398e6: ending tablet copy session 6251da0ffb45473fb96b1fb0abd9d14b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6251da0ffb45473fb96b1fb0abd9d14b
I20260811 07:39:13.286586 25972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:13.288157 26100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.288235 26103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.288695 26101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:13.288913 25972 server_base.cc:1034] running on GCE node
I20260811 07:39:13.289068 25972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:13.289350 25972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:13.290513 25972 hybrid_clock.cc:648] HybridClock initialized: now 1786433953290489 us; error 41 us; skew 500 ppm
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:13.292268 25972 init.cc:377] Logged in from keytab as oryx/127.18.72.253@KRBTEST.COM (short username oryx)
I20260811 07:39:13.292724 25972 webserver.cc:460] Webserver started at http://127.18.72.253:35811/ using document root <none> and password file <none>
I20260811 07:39:13.292913 25972 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:13.292956 25972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:13.294308 25972 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:13.295315 26111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.295580 25972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:13.295653 25972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
uuid: "6251da0ffb45473fb96b1fb0abd9d14b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "6c649e8a11b105cb2dcc8c763b4abf95"
server_key_iv: "15a8a2be2b04105d0673ce10bf8a6653"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.295735 25972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:13.311050 25972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:13.313277 25972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:13.318545 25972 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.253:46723
I20260811 07:39:13.319176 25972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
I20260811 07:39:13.321040 26176 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:39:13.322039 26176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Bootstrap starting.
I20260811 07:39:13.324232 26175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.253:46723 every 8 connection(s)
I20260811 07:39:13.332700 26176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:39:13.333081 26176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: Bootstrap complete.
I20260811 07:39:13.333673 26176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } }
I20260811 07:39:13.333877 26176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6251da0ffb45473fb96b1fb0abd9d14b, State: Initialized, Role: LEARNER
I20260811 07:39:13.333983 26176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } }
I20260811 07:39:13.334326 26179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } }
I20260811 07:39:13.334530 26176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:13.334529 26179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: This master's current role is: LEARNER
I20260811 07:39:13.337659 26193 catalog_manager.cc:1259] Loaded cluster ID: e73456e2e8d0444db1bd5ea14054a285
I20260811 07:39:13.337781 26193 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: loading cluster ID for follower catalog manager: success
I20260811 07:39:13.340029 26193 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: acquiring CA information for follower catalog manager: success
I20260811 07:39:13.341605 26193 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 07:39:13.387440 26091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:13.387749 26091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:13.387835 26091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:13.391808 26091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:13.391906 26091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:13.391971 26091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:13.392030 26091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:13.392067 26091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:13.397926 26091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:45797,127.18.72.253:46723,127.18.72.252:35873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.252.keytab
--principal=oryx/127.18.72.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.252:35873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.72.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:13.399106 26091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:13.400282 26091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:13.408859 26198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.408883 26199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:13.409044 26091 server_base.cc:1034] running on GCE node
W20260811 07:39:13.408874 26201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:13.409775 26091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:13.411008 26091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:13.412204 26091 hybrid_clock.cc:648] HybridClock initialized: now 1786433953412183 us; error 28 us; skew 500 ppm
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:13.415177 26091 init.cc:377] Logged in from keytab as oryx/127.18.72.252@KRBTEST.COM (short username oryx)
I20260811 07:39:13.416261 26091 webserver.cc:460] Webserver started at http://127.18.72.252:40375/ using document root <none> and password file <none>
I20260811 07:39:13.416877 26091 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:13.416977 26091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:13.417289 26091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:13.419613 26091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data/instance:
uuid: "23d3225852b24df2ab3740deb6e1d15b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "0d32969d24292996615de56941453a7d"
server_key_iv: "5f72a70038104d5ac509ce98045c0232"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.420285 26091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal/instance:
uuid: "23d3225852b24df2ab3740deb6e1d15b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "0d32969d24292996615de56941453a7d"
server_key_iv: "5f72a70038104d5ac509ce98045c0232"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.424304 26091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 07:39:13.426939 26209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.428437 26091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:13.428601 26091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
uuid: "23d3225852b24df2ab3740deb6e1d15b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "0d32969d24292996615de56941453a7d"
server_key_iv: "5f72a70038104d5ac509ce98045c0232"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.428723 26091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:13.440027 26091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:13.443571 26091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:13.443800 26091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:13.451586 26091 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.252:35873
I20260811 07:39:13.451869 26264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.252:35873 every 8 connection(s)
I20260811 07:39:13.452728 26091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
I20260811 07:39:13.453663 25663 heartbeater.cc:503] Master 127.18.72.254:45797 was elected leader, sending a full tablet report...
I20260811 07:39:13.456050 26265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:13.456323 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 26091
I20260811 07:39:13.462471 26265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.72.252@KRBTEST.COM
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.252@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:13.474258 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.457795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.18.72.252:35873)
0811 07:39:13.458344 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.458368 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.458415 (+ 47us) server_negotiation.cc:407] Connection header received
0811 07:39:13.459471 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.459498 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.460002 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.460414 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.461529 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.463176 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.464538 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.465277 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.465877 (+ 600us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.468349 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.468356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.468368 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.468399 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.470906 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.471387 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.471393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.471399 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.471515 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.471813 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.471816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.471818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.472372 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.472601 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.472934 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.473232 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":144,"threads_started":1}
I20260811 07:39:13.475284 26268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.464651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:36768)
0811 07:39:13.466076 (+ 1425us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.466123 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.466258 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.466559 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.466564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.466920 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.467046 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.467063 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.467934 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.467939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.468824 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.468875 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.469361 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.469364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.469482 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.469938 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.469958 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.471637 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.473669 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.473676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.473680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.473926 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.474219 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.474221 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.474222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.474294 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.474770 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.474776 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.474885 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.475010 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.475137 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1255,"thread_start_us":114,"threads_started":1}
I20260811 07:39:13.475395 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.465089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.18.72.254:45797)
0811 07:39:13.465267 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.465272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.466207 (+ 935us) server_negotiation.cc:407] Connection header received
0811 07:39:13.466357 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.466360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.466408 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.466507 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.467202 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.467747 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.468986 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.469226 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.469356 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.471828 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.471831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.471834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.471862 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.473559 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.474028 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.474031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.474033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.474125 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.474417 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.474420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.474422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.474603 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.474714 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.475101 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.475246 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 07:39:13.476481 26265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.252@KRBTEST.COM for oryx/127.18.72.253@KRBTEST.COM
I20260811 07:39:13.484117 26268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.476987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.0.0.1:60458)
0811 07:39:13.477097 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.477112 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.477186 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.477541 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.477543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.477763 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.477904 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.477925 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.478782 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.478786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.479317 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.479326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.479725 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.479727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.479799 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.480216 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.480228 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.481392 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.482887 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.482889 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.482891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.483065 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.483306 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.483308 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.483309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.483378 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.483690 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.483692 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.483783 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.483867 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.484014 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 07:39:13.484216 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.477098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.18.72.253:46723)
0811 07:39:13.477340 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.477344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.477356 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:13.477396 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.477398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.477438 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.477514 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.478037 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.478667 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.479419 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.479630 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.479705 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.481522 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.481525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.481527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.481548 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.482790 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.483144 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.483147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.483148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.483217 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.483456 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.483458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.483459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.483609 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.483681 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.483986 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.484080 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
I20260811 07:39:13.485097 26265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.252@KRBTEST.COM for oryx/127.18.72.252@KRBTEST.COM
I20260811 07:39:13.492261 26268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.485561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.0.0.1:39086)
0811 07:39:13.485644 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.485659 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.485728 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.485921 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.485923 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.486041 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.486102 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.486106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.486762 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.486765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.487244 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.487251 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.488146 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.488149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.488204 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.488552 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.488562 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.489617 (+ 1055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.490990 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.490992 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.490994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.491175 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.491425 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.491427 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.491428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.491498 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.491824 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.491826 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.491968 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.492054 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.492151 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 07:39:13.492374 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.485600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.18.72.252:35873)
0811 07:39:13.485709 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.485712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.485724 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:13.485815 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.485817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.485850 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.485908 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.486185 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.486667 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.487340 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.488049 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.488132 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.489735 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.489737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.489739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.489759 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.490898 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.491264 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.491267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.491268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.491345 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.491575 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.491577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.491578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.491734 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.491832 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.492132 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.492229 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:39:13.495299 26265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Bootstrap starting.
I20260811 07:39:13.497814 26265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:13.498664 26265 log.cc:824] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:13.500703 26265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: No bootstrap required, opened a new log
I20260811 07:39:13.503721 26265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } }
I20260811 07:39:13.504024 26265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:13.504112 26265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d3225852b24df2ab3740deb6e1d15b, State: Initialized, Role: FOLLOWER
I20260811 07:39:13.504551 26265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } }
I20260811 07:39:13.505043 26271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } } }
I20260811 07:39:13.505163 26271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:39:13.505777 26265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: configured and running, proceeding with master startup.
W20260811 07:39:13.510728 26286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:39:13.510828 26286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:39:13.513875 26091 master_runner.cc:428] Detected that this master 23d3225852b24df2ab3740deb6e1d15b is joining an existing cluster
I20260811 07:39:13.513926 26091 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.72.252:35873
I20260811 07:39:13.525683 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.516226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.0.0.1:39096)
0811 07:39:13.516975 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.516992 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.517087 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.517426 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.517429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.517676 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.517812 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.517820 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.518780 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.518783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.519650 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.519661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.520277 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.520281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.520411 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.520963 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.520978 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.521435 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.524308 (+ 2873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.524311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.524312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.524501 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.524814 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.524816 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.524817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.524892 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.525298 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.525300 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.525372 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.525450 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.525545 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":22,"thread_start_us":121,"threads_started":1}
I20260811 07:39:13.525846 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.516951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39096 (local address 127.18.72.252:35873)
0811 07:39:13.517051 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.517056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.517068 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:13.517175 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.517179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.517222 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.517300 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.517924 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.518670 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.519765 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.520145 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.520261 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.521589 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.521597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.521601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.521645 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.524189 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.524598 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.524602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.524606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.524700 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.524978 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.524982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.524984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.525203 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.525302 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.525527 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.525642 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":31,"spinlock_wait_cycles":13056}
I20260811 07:39:13.528028 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 25517
I20260811 07:39:13.544503 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:33509
--local_ip_for_outbound_sockets=127.18.72.193
--tserver_master_addrs=127.18.72.254:45797
--webserver_port=41963
--webserver_interface=127.18.72.193
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab
--principal=oryx/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723,127.18.72.252:35873 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:39:13.545313 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.516624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60464 (local address 127.18.72.253:46723)
0811 07:39:13.516811 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.516816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.517194 (+ 378us) server_negotiation.cc:407] Connection header received
0811 07:39:13.517412 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.517416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.517474 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.517529 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.518188 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.519063 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.520484 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.520841 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.520923 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.540982 (+ 20059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.540990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.540996 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.541033 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.543171 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.543813 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.543818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.543820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.543945 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.544281 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.544285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.544287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.544548 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.544652 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.544996 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.545142 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 07:39:13.545815 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.516290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.0.0.1:60464)
0811 07:39:13.517108 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.517122 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.517264 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.517709 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.517712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.517931 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.518048 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.518055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.519205 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.519208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.520332 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.520344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.521082 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.521086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.521181 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.521644 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.521658 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.540751 (+ 19093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.543331 (+ 2580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.543334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.543337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.543675 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.544075 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.544077 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.544079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.544168 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.544701 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.544704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.544807 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.545473 (+ 666us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.545630 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":90,"spinlock_wait_cycles":5760,"thread_start_us":135,"threads_started":1}
I20260811 07:39:13.547430 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.516790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36770 (local address 127.18.72.254:45797)
0811 07:39:13.516907 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.516912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.517396 (+ 484us) server_negotiation.cc:407] Connection header received
0811 07:39:13.517624 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.517627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.517678 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.517730 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.518348 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.519126 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.520746 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.521104 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.521213 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.541129 (+ 19916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.541134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.541136 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.541170 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.543316 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.543835 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.543838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.543839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.543945 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.546488 (+ 2543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.546492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.546493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.546729 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.546806 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.547169 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.547291 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:39:13.547446 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.516711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:36770)
0811 07:39:13.517317 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.517330 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.517502 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.517932 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.517935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.518113 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.518212 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.518219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.519284 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.519287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.520610 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.520619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.521352 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.521358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.521452 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.540477 (+ 19025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.540497 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.540954 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.543432 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.543435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.543437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.543720 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.546260 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.546263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.546265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.546369 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.546877 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.546882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.546986 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.547111 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.547261 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":46,"thread_start_us":80,"threads_started":1}
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433952, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.254@KRBTEST.COM for oryx/127.18.72.252@KRBTEST.COM
I20260811 07:39:13.561419 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.551779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.0.0.1:39102)
0811 07:39:13.551958 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.551971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.552047 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.552375 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.552378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.552581 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.552689 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.552695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.553804 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.553808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.554671 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.554682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.555338 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.555342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.555460 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.555986 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.556001 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.557686 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.559681 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.559684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.559687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.560128 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.560425 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.560429 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.560431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.560521 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.560918 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.560921 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.561016 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.561109 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.561248 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 07:39:13.561718 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.551879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.18.72.252:35873)
0811 07:39:13.552035 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.552042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.552058 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:39:13.552163 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.552168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.552238 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.552345 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.552814 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.553662 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.554801 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.555203 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.555345 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.557833 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.557837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.557840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.557871 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.559513 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.560230 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.560234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.560236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.560319 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.560623 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.560626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.560629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.560826 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.560918 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.561194 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.561307 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 07:39:13.562747 25449 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.72.252:35873: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b"
member_type: NON_VOTER
last_known_addr {
host: "127.18.72.252"
port: 35873
}
attrs {
promote: true
}
}
dest_uuid: "7461a49810784a64b6c0e21a9dd398e6"
cas_config_opid_index: 5
I20260811 07:39:13.563393 25449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } }
I20260811 07:39:13.565061 26084 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 23d3225852b24df2ab3740deb6e1d15b (127.18.72.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.566111 26238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:13.566499 26084 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.72.252:35873
I20260811 07:39:13.566663 26084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.566747 26084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:13.567409 26091 master.cc:537] Master@127.18.72.252:35873 shutting down...
I20260811 07:39:13.567556 26238 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Refusing update from remote peer 7461a49810784a64b6c0e21a9dd398e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 07:39:13.568223 26082 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 07:39:13.570039 25438 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 -> Peer 23d3225852b24df2ab3740deb6e1d15b (127.18.72.252:35873): Couldn't send request to peer 23d3225852b24df2ab3740deb6e1d15b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 07:39:13.573246 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.565465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60470 (local address 127.18.72.253:46723)
0811 07:39:13.565573 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.565577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.568299 (+ 2722us) server_negotiation.cc:407] Connection header received
0811 07:39:13.568497 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.568500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.568559 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.568625 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:13.569038 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.569897 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.571483 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.572202 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.572280 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.572468 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.573051 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.573126 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:39:13.573740 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.565244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.0.0.1:60470)
0811 07:39:13.568229 (+ 2985us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.568243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.568379 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.568793 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.568796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.568813 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:13.568907 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.568912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.570097 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.570101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.571330 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.571342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.572786 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.572789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.572885 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.572930 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.573525 (+ 595us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.573609 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2912}
I20260811 07:39:13.574290 26143 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 LEARNER]: Refusing update from remote peer 7461a49810784a64b6c0e21a9dd398e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 07:39:13.574806 26296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 07:39:13.575156 26091 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:13.575461 26091 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:13.575563 26091 tablet_replica.cc:330] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: stopping tablet replica
I20260811 07:39:13.576990 26143 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 23d3225852b24df2ab3740deb6e1d15b (127.18.72.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.578866 26179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7461a49810784a64b6c0e21a9dd398e6. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.578972 26179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: This master's current role is: LEARNER
I20260811 07:39:13.579049 26179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: NON_VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: true } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.579107 26179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: This master's current role is: LEARNER
I20260811 07:39:13.580006 26082 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: attempting to promote NON_VOTER 6251da0ffb45473fb96b1fb0abd9d14b to VOTER
I20260811 07:39:13.580451 26082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } }
W20260811 07:39:13.581914 26211 connection.cc:441] server connection from 127.0.0.1:39102 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39102 (request call id 4) could send its response
I20260811 07:39:13.583174 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.582870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.0.0.1:39104)
0811 07:39:13.583009 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.583083 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:35873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 07:39:13.583387 26140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 LEARNER]: Refusing update from remote peer 7461a49810784a64b6c0e21a9dd398e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260811 07:39:13.583778 25438 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 -> Peer 23d3225852b24df2ab3740deb6e1d15b (127.18.72.252:35873): Couldn't send request to peer 23d3225852b24df2ab3740deb6e1d15b. Status: Network error: Client connection negotiation failed: client connection to 127.18.72.252:35873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 07:39:13.584005 26299 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 07:39:13.585743 26084 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6251da0ffb45473fb96b1fb0abd9d14b (127.18.72.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.587298 26084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.587396 26084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:13.587494 26084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.587554 26084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:13.588438 26140 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6251da0ffb45473fb96b1fb0abd9d14b (127.18.72.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.590011 26298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.590175 26298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:39:13.594960 26091 master.cc:552] Master@127.18.72.252:35873 shutdown complete.
I20260811 07:39:13.595033 26091 master_runner.cc:305] Clearing existing system tablet
I20260811 07:39:13.595299 26091 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:13.595361 26091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:13.596699 26091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:39:13.597621 26303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.597862 26091 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:13.597988 26091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 07:39:13.598022 26091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
uuid: "23d3225852b24df2ab3740deb6e1d15b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "0d32969d24292996615de56941453a7d"
server_key_iv: "5f72a70038104d5ac509ce98045c0232"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.600096 26091 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:13.602437 26091 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 07:39:13.602527 26091 log.cc:1196] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 07:39:13.602777 26091 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Deleting consensus metadata
I20260811 07:39:13.602983 26091 master_runner.cc:315] Copying system tablet from 127.18.72.254:45797
I20260811 07:39:13.604219 26091 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: tablet copy: Beginning tablet copy session from remote peer at address 127.18.72.254:45797
I20260811 07:39:13.614245 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.604872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.18.72.254:45797)
0811 07:39:13.605002 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.605006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.605919 (+ 913us) server_negotiation.cc:407] Connection header received
0811 07:39:13.606040 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.606043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.606097 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.606185 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.606723 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.607596 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.608757 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.609081 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.609185 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.610456 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.610460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.610463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.610492 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.612246 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.612750 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.612753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.612755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.612848 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.613150 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.613153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.613154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.613403 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.613517 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.613954 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.614079 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:39:13.614284 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.604757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:36784)
0811 07:39:13.605806 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.605822 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.605953 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.606219 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.606222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.606446 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.606564 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.606572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.607706 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.607709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.608615 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.608626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.609203 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.609207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.609294 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.609862 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.609878 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.610305 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.612383 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.612387 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.612389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.612642 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.612945 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.612948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.612950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.613044 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.613566 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.613569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.613667 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.613864 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.614014 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"thread_start_us":88,"threads_started":1}
I20260811 07:39:13.614975 25471 tablet_copy_service.cc:140] P 7461a49810784a64b6c0e21a9dd398e6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 23d3225852b24df2ab3740deb6e1d15b ({username='oryx', principal='oryx/127.18.72.252@KRBTEST.COM'} at 127.0.0.1:36784)
I20260811 07:39:13.615068 25471 tablet_copy_service.cc:161] P 7461a49810784a64b6c0e21a9dd398e6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 23d3225852b24df2ab3740deb6e1d15b at {username='oryx', principal='oryx/127.18.72.252@KRBTEST.COM'} at 127.0.0.1:36784: session id = 23d3225852b24df2ab3740deb6e1d15b-00000000000000000000000000000000
I20260811 07:39:13.616181 25471 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 07:39:13.617693 26091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 07:39:13.620203 26091 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: tablet copy: Starting download of 0 data blocks...
I20260811 07:39:13.620380 26091 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: tablet copy: Starting download of 1 WAL segments...
I20260811 07:39:13.621882 26091 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 07:39:13.623615 25471 tablet_copy_service.cc:342] P 7461a49810784a64b6c0e21a9dd398e6: Request end of tablet copy session 23d3225852b24df2ab3740deb6e1d15b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.72.252@KRBTEST.COM'} at 127.0.0.1:36784
I20260811 07:39:13.623692 25471 tablet_copy_service.cc:433] P 7461a49810784a64b6c0e21a9dd398e6: ending tablet copy session 23d3225852b24df2ab3740deb6e1d15b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 23d3225852b24df2ab3740deb6e1d15b
I20260811 07:39:13.631395 26091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:13.633165 26315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.633165 26312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.636904 26313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:13.637195 26091 server_base.cc:1034] running on GCE node
I20260811 07:39:13.637363 26091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:13.637665 26091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:13.638828 26091 hybrid_clock.cc:648] HybridClock initialized: now 1786433953638814 us; error 29 us; skew 500 ppm
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:13.640556 26091 init.cc:377] Logged in from keytab as oryx/127.18.72.252@KRBTEST.COM (short username oryx)
I20260811 07:39:13.641067 26091 webserver.cc:460] Webserver started at http://127.18.72.252:38205/ using document root <none> and password file <none>
I20260811 07:39:13.641259 26091 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:13.641314 26091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:13.642771 26091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:13.643842 26323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.644209 26091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:13.644292 26091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
uuid: "23d3225852b24df2ab3740deb6e1d15b"
format_stamp: "Formatted at 2026-08-11 07:39:13 on dist-test-slave-zkpd"
server_key: "0d32969d24292996615de56941453a7d"
server_key_iv: "5f72a70038104d5ac509ce98045c0232"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.644375 26091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:13.666644 26091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:13.668076 25813 heartbeater.cc:503] Master 127.18.72.254:45797 was elected leader, sending a full tablet report...
I20260811 07:39:13.669116 26091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:13.674305 26091 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.252:35873
I20260811 07:39:13.674335 26378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.252:35873 every 8 connection(s)
I20260811 07:39:13.674908 26091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
I20260811 07:39:13.676801 26379 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:39:13.677826 26379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Bootstrap starting.
I20260811 07:39:13.688723 26379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:39:13.689049 26379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Bootstrap complete.
I20260811 07:39:13.689684 26379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } }
I20260811 07:39:13.689828 26379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 23d3225852b24df2ab3740deb6e1d15b, State: Initialized, Role: LEARNER
I20260811 07:39:13.689956 26379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } }
I20260811 07:39:13.690297 26386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:13.690407 26379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:13.690490 26386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: This master's current role is: LEARNER
I20260811 07:39:13.693737 26397 catalog_manager.cc:1259] Loaded cluster ID: e73456e2e8d0444db1bd5ea14054a285
I20260811 07:39:13.693843 26397 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: loading cluster ID for follower catalog manager: success
W20260811 07:39:13.695852 26295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260811 07:39:13.696089 26397 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: acquiring CA information for follower catalog manager: success
W20260811 07:39:13.696185 26295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:13.696254 26295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 07:39:13.697470 26397 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 07:39:13.700730 26295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:13.700788 26295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:13.700888 26295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:13.706072 26295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab
--principal=oryx/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:33509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=41963
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723,127.18.72.252:35873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:13.707073 26295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:13.708153 26295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:13.717805 26399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.717818 26402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.717881 26400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:13.718065 26295 server_base.cc:1034] running on GCE node
I20260811 07:39:13.718487 26295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:13.719192 26295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:13.720379 26295 hybrid_clock.cc:648] HybridClock initialized: now 1786433953720364 us; error 26 us; skew 500 ppm
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:13.723054 26295 init.cc:377] Logged in from keytab as oryx/127.18.72.193@KRBTEST.COM (short username oryx)
I20260811 07:39:13.724001 26295 webserver.cc:460] Webserver started at http://127.18.72.193:41963/ using document root <none> and password file <none>
I20260811 07:39:13.724787 26295 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:13.724902 26295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:13.728736 26295 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 07:39:13.730679 26409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.732146 26295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:13.732237 26295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "08870894ac9541618bd48a2fe3dae028"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "f71837484c02967791d8b80920dfecc1"
server_key_iv: "57bd38d57547cd0ff88b1d3cc25ecf2e"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.732854 26295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:13.754858 26295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:13.758535 26295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:13.758688 26295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:13.759200 26295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:13.760128 26295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:13.760176 26295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.760216 26295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:13.760237 26295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.768886 26295 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:33509
I20260811 07:39:13.768977 26532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:33509 every 8 connection(s)
I20260811 07:39:13.769897 26295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:13.773381 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 26295
I20260811 07:39:13.773586 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 25668
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.253@KRBTEST.COM
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.252@KRBTEST.COM
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:13.788607 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.771704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:52805 (local address 127.18.72.253:46723)
0811 07:39:13.772372 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.772377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.776483 (+ 4106us) server_negotiation.cc:407] Connection header received
0811 07:39:13.776582 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.776585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.776642 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.776729 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.777440 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.777968 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.779616 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.779993 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.780069 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.783668 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.783672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.783675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.783702 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.786256 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.786727 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.786730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.786732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.786818 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.787253 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.787257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.787258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.787478 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.787553 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.788316 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.788442 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581}
I20260811 07:39:13.788610 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.772312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:44973 (local address 127.18.72.252:35873)
0811 07:39:13.772601 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.772606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.776138 (+ 3532us) server_negotiation.cc:407] Connection header received
0811 07:39:13.776307 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.776315 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.776392 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.776516 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.778071 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.779150 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.780312 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.780659 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.780748 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.784096 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.784103 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.784112 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.784145 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.785983 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.786698 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.786704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.786710 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.786803 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.787253 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.787257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.787259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.787478 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.787558 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.788322 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.788444 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":79,"threads_started":1}
I20260811 07:39:13.789093 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.771816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:39953 (local address 127.18.72.254:45797)
0811 07:39:13.771991 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:13.771996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:13.773130 (+ 1134us) server_negotiation.cc:407] Connection header received
0811 07:39:13.774565 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:13.774572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:13.774652 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:13.774741 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:13.778124 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.779257 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.781599 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.782099 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.782192 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.784863 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:13.784870 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:13.784875 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:13.784909 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:13.786785 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.787494 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.787501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.787505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.787601 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:13.787996 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:13.788002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:13.788007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:13.788248 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:13.788334 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:13.788712 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 07:39:13.788862 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:39:13.790445 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.772235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.18.72.193:44973)
0811 07:39:13.776057 (+ 3822us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.776069 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.776152 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.777480 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.777486 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.777747 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.777916 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.777930 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.779284 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.779287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.780181 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.780190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.780898 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.780900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.781016 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.781534 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.781548 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.783903 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.786125 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.786132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.786149 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.786576 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.786921 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.786924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.786926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.787138 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.787629 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.787635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.788012 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.788905 (+ 893us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.789258 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3740,"mutex_wait_us":40,"spinlock_wait_cycles":48128,"thread_start_us":152,"threads_started":1}
I20260811 07:39:13.790709 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.772235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.193:39953)
0811 07:39:13.772948 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.773035 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.774313 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.775281 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.775297 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.776060 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.777306 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.777322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.779432 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.779438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.781126 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.781143 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.782393 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.782399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.782537 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.783070 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.783090 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.784699 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.786918 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.786921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.786923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.787362 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.787717 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.787722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.787727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.787821 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.788413 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.788418 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.788522 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.789298 (+ 776us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.789465 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"spinlock_wait_cycles":3328,"thread_start_us":142,"threads_started":1}
I20260811 07:39:13.792254 26534 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45797
I20260811 07:39:13.792522 26534 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:13.792714 26533 heartbeater.cc:347] Connected to a master server at 127.18.72.252:35873
I20260811 07:39:13.792820 26533 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:13.790453 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:13.772235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.18.72.193:52805)
0811 07:39:13.776376 (+ 4141us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:13.776393 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:13.776496 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:13.776743 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:13.776745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:13.776954 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:13.777298 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.777312 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.778156 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.778160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.779483 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:13.779492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:13.780280 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:13.780287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:13.780472 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:13.781171 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:13.781196 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:13.783500 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:13.786372 (+ 2872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.786375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.786377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.786616 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.786925 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:13.786927 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:13.786928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:13.787138 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:13.787629 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:13.787635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:13.788012 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 07:39:13.789104 (+ 1092us) client_negotiation.cc:240] Negotiation successful
0811 07:39:13.789383 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4035,"mutex_wait_us":140,"spinlock_wait_cycles":28672,"thread_start_us":105,"threads_started":1}
I20260811 07:39:13.793092 26533 heartbeater.cc:511] Master 127.18.72.252:35873 requested a full tablet report, sending...
I20260811 07:39:13.793089 26534 heartbeater.cc:511] Master 127.18.72.254:45797 requested a full tablet report, sending...
I20260811 07:39:13.794425 25449 ts_manager.cc:194] Re-registered known tserver with Master: 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509)
I20260811 07:39:13.795030 26342 ts_manager.cc:194] Registered new tserver with Master: 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509)
I20260811 07:39:13.795169 25449 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:39953
I20260811 07:39:13.802125 26535 heartbeater.cc:347] Connected to a master server at 127.18.72.253:46723
I20260811 07:39:13.802224 26535 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:13.802453 26535 heartbeater.cc:511] Master 127.18.72.253:46723 requested a full tablet report, sending...
I20260811 07:39:13.803665 26121 ts_manager.cc:194] Registered new tserver with Master: 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509)
I20260811 07:39:13.807760 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:45417
--local_ip_for_outbound_sockets=127.18.72.194
--tserver_master_addrs=127.18.72.254:45797
--webserver_port=46191
--webserver_interface=127.18.72.194
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab
--principal=oryx/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723,127.18.72.252:35873 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:39:13.904448 25967 heartbeater.cc:503] Master 127.18.72.254:45797 was elected leader, sending a full tablet report...
W20260811 07:39:13.927758 26541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:13.928016 26541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:13.928097 26541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:13.932132 26541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:13.932188 26541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:13.932273 26541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:13.938087 26541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab
--principal=oryx/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:45417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=46191
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723,127.18.72.252:35873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:13.939133 26541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:13.940204 26541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:13.950053 26547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:13.950048 26546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:13.950356 26541 server_base.cc:1034] running on GCE node
W20260811 07:39:13.950053 26549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:13.950748 26541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:13.951366 26541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:13.952529 26541 hybrid_clock.cc:648] HybridClock initialized: now 1786433953952515 us; error 23 us; skew 500 ppm
Aug 11 07:39:13 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:13.955227 26541 init.cc:377] Logged in from keytab as oryx/127.18.72.194@KRBTEST.COM (short username oryx)
I20260811 07:39:13.956094 26541 webserver.cc:460] Webserver started at http://127.18.72.194:46191/ using document root <none> and password file <none>
I20260811 07:39:13.956632 26541 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:13.956705 26541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:13.960366 26541 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 07:39:13.962347 26558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.963568 26541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:13.963690 26541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "d3386466d6de4fb29244bcf1adc944d6"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "544f9682b632e29cf9cdae1fd0d0c087"
server_key_iv: "0949a1bcc61577445b44a387fad6075d"
server_key_version: "encryptionkey@0"
I20260811 07:39:13.964304 26541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:13.990057 26541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:13.993507 26541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:13.993726 26541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:13.994391 26541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:13.995254 26541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:13.995334 26541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:13.995398 26541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:13.995422 26541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:14.005545 26541 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:45417
I20260811 07:39:14.005625 26680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:45417 every 8 connection(s)
I20260811 07:39:14.006587 26541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:14.015254 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 26541
I20260811 07:39:14.015461 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 25819
Aug 11 07:39:14 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
Aug 11 07:39:14 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.253@KRBTEST.COM
Aug 11 07:39:14 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433953, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.252@KRBTEST.COM
I20260811 07:39:14.023187 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.008423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:33465 (local address 127.18.72.254:45797)
0811 07:39:14.008580 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.008585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.010099 (+ 1514us) server_negotiation.cc:407] Connection header received
0811 07:39:14.010658 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.010661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.010718 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.010774 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.012039 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.012613 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.014016 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.014247 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.014323 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.018412 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.018416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.018419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.018446 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.020891 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.021395 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.021400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.021404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.021481 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.021876 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.021882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.021886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.022116 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.022184 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.022830 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.022961 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 07:39:14.023554 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.008505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:57859 (local address 127.18.72.253:46723)
0811 07:39:14.008655 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.008661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.009612 (+ 951us) server_negotiation.cc:407] Connection header received
0811 07:39:14.010636 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.010639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.010698 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.010777 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.012121 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.012637 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.014428 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.014786 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.014902 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.018135 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.018142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.018149 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.018190 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.020542 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.021357 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.021365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.021370 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.021487 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.021927 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.021934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.021939 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.022259 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.022360 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.022858 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.023020 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:39:14.024813 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.008900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.18.72.194:57859)
0811 07:39:14.009433 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.009483 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.010434 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.010940 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.010947 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.011381 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:14.011866 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.011879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.012778 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.012781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.013890 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.013899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.015109 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.015113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.015248 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.015874 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:14.015912 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:14.017894 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:14.020673 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.020682 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.020704 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.021206 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.021630 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.021636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.021641 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.021788 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.022454 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:14.022459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:14.022592 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.023969 (+ 1377us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.024160 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":61,"thread_start_us":138,"threads_started":1}
I20260811 07:39:14.025070 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.008900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.194:33465)
0811 07:39:14.009985 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.009999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.010452 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.010968 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.010970 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.011429 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:14.011867 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.011885 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.012763 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.012768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.013883 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.013893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.014391 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.014395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.014604 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.015244 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:14.015266 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:14.018263 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:14.021012 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.021017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.021022 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.021292 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.021584 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.021589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.021593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.021756 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.022259 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:14.022267 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:14.022569 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.023406 (+ 837us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.023739 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":270,"spinlock_wait_cycles":19584,"thread_start_us":139,"threads_started":1}
I20260811 07:39:14.027509 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.008900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.18.72.194:33717)
0811 07:39:14.009933 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.009947 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.010451 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.010955 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.010961 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.011451 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:14.011867 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.011885 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.013720 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.013723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.014407 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.014416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.014987 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.014991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.015113 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.016543 (+ 1430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:14.016562 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:14.018624 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:14.025828 (+ 7204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.025831 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.025834 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.026075 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.026479 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.026481 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.026483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.026555 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.026961 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:14.026964 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:14.027063 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.027171 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.027328 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":83,"spinlock_wait_cycles":47616,"thread_start_us":162,"threads_started":1}
I20260811 07:39:14.027516 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.008932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:33717 (local address 127.18.72.252:35873)
0811 07:39:14.009047 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.009051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.010080 (+ 1029us) server_negotiation.cc:407] Connection header received
0811 07:39:14.010659 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.010662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.010720 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.010782 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.012774 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.013583 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.014526 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.014874 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.014967 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.018833 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.018836 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.018838 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.018861 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.025703 (+ 6842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.026246 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.026250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.026252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.026376 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.026654 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.026657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.026659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.026856 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.026972 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.027239 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.027375 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 07:39:14.031829 26681 heartbeater.cc:347] Connected to a master server at 127.18.72.252:35873
I20260811 07:39:14.031956 26682 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45797
I20260811 07:39:14.032075 26683 heartbeater.cc:347] Connected to a master server at 127.18.72.253:46723
I20260811 07:39:14.032166 26681 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:14.032167 26683 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:14.032195 26682 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:14.032895 26683 heartbeater.cc:511] Master 127.18.72.253:46723 requested a full tablet report, sending...
I20260811 07:39:14.032891 26681 heartbeater.cc:511] Master 127.18.72.252:35873 requested a full tablet report, sending...
I20260811 07:39:14.032892 26682 heartbeater.cc:511] Master 127.18.72.254:45797 requested a full tablet report, sending...
I20260811 07:39:14.034076 25449 ts_manager.cc:194] Re-registered known tserver with Master: d3386466d6de4fb29244bcf1adc944d6 (127.18.72.194:45417)
I20260811 07:39:14.034148 26121 ts_manager.cc:194] Registered new tserver with Master: d3386466d6de4fb29244bcf1adc944d6 (127.18.72.194:45417)
I20260811 07:39:14.034148 26342 ts_manager.cc:194] Registered new tserver with Master: d3386466d6de4fb29244bcf1adc944d6 (127.18.72.194:45417)
I20260811 07:39:14.034916 25449 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:33465
I20260811 07:39:14.035172 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:33449
--local_ip_for_outbound_sockets=127.18.72.195
--tserver_master_addrs=127.18.72.254:45797
--webserver_port=35125
--webserver_interface=127.18.72.195
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab
--principal=oryx/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723,127.18.72.252:35873 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:14.152933 26689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:14.153196 26689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:14.153271 26689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:14.157218 26689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:14.157305 26689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:14.157397 26689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:14.163324 26689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab
--principal=oryx/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:33449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=35125
--tserver_master_addrs=127.18.72.254:45797,127.18.72.253:46723,127.18.72.252:35873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:14.164577 26689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:14.164595 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.160657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.18.72.252:35873)
0811 07:39:14.160810 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.160814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.160889 (+ 75us) server_negotiation.cc:407] Connection header received
0811 07:39:14.161041 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.161044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.161098 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.161182 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:14.161457 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.161959 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.163507 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.164058 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.164146 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.164276 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.164430 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.164495 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 07:39:14.164640 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.160510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.0.0.1:39114)
0811 07:39:14.160787 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.160803 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.160928 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.161202 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.161206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.161226 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:14.161351 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.161358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.162064 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.162067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.163322 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.163338 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.164181 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.164185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.164283 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.164312 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.164407 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.164489 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":70,"threads_started":1}
I20260811 07:39:14.165254 26351 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 07:39:14.165647 26386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7461a49810784a64b6c0e21a9dd398e6. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:14.165807 26386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: This master's current role is: LEARNER
I20260811 07:39:14.165897 26689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:14.175853 26699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:14.175858 26702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:14.176155 26689 server_base.cc:1034] running on GCE node
W20260811 07:39:14.175917 26700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:14.176669 26689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:14.177388 26689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:14.178578 26689 hybrid_clock.cc:648] HybridClock initialized: now 1786433954178545 us; error 43 us; skew 500 ppm
Aug 11 07:39:14 dist-test-slave-zkpd krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433954, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:14.181422 26689 init.cc:377] Logged in from keytab as oryx/127.18.72.195@KRBTEST.COM (short username oryx)
I20260811 07:39:14.182319 26689 webserver.cc:460] Webserver started at http://127.18.72.195:35125/ using document root <none> and password file <none>
I20260811 07:39:14.182878 26689 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:14.183002 26689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:14.186796 26689 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:14.188850 26712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:14.190100 26689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:14.190238 26689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "25c89f1b811e4b08a8e691bf1fbce46a"
format_stamp: "Formatted at 2026-08-11 07:39:12 on dist-test-slave-zkpd"
server_key: "8fbc0258d091513aab9456b6cfc860db"
server_key_iv: "d19f0fe0d73c83384ac839f17d5fba29"
server_key_version: "encryptionkey@0"
I20260811 07:39:14.190873 26689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:14.204892 26689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:14.208114 26689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:14.208310 26689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:14.208889 26689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:14.209746 26689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:14.209816 26689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:14.209890 26689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:14.209932 26689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:14.219244 26689 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:33449
I20260811 07:39:14.219297 26833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:33449 every 8 connection(s)
I20260811 07:39:14.220316 26689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:14.222159 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 26689
Aug 11 07:39:14 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433954, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
Aug 11 07:39:14 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433954, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.252@KRBTEST.COM
Aug 11 07:39:14 dist-test-slave-zkpd krb5kdc[25402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433954, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.253@KRBTEST.COM
I20260811 07:39:14.239530 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.224335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.18.72.253:46723)
0811 07:39:14.226079 (+ 1744us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.226083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.226315 (+ 232us) server_negotiation.cc:407] Connection header received
0811 07:39:14.226571 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.226575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.226627 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.226688 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.230417 (+ 3729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.231153 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.232343 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.232728 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.232805 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.234845 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.234849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.234852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.234876 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.237227 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.237744 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.237800 (+ 56us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.237802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.237891 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.238220 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.238226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.238229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.238470 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.238597 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.239211 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.239344 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1668,"thread_start_us":84,"threads_started":1}
I20260811 07:39:14.239965 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.222404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:43557 (local address 127.18.72.254:45797)
0811 07:39:14.223008 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.223013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.223518 (+ 505us) server_negotiation.cc:407] Connection header received
0811 07:39:14.225618 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.225624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.225681 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.225735 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.227009 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.227816 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.229522 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.229859 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.229926 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.235015 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.235019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.235021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.235047 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.237531 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.238175 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.238179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.238181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.238269 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.238669 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.238673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.238674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.238905 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.238986 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.239659 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.239770 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502}
I20260811 07:39:14.240818 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.222919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:43803 (local address 127.18.72.252:35873)
0811 07:39:14.223059 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.223064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.224218 (+ 1154us) server_negotiation.cc:407] Connection header received
0811 07:39:14.225020 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.225025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.225090 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.225148 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.228503 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.229209 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.233231 (+ 4022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.233558 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.233630 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.236523 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.236527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.236530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.236557 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.238585 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.239075 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.239079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.239081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.239158 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.239485 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.239490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.239492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.239702 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.239788 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.240545 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.240663 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 07:39:14.241019 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.222846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:58563 (local address 127.18.72.253:46723)
0811 07:39:14.223137 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.223141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.224014 (+ 873us) server_negotiation.cc:407] Connection header received
0811 07:39:14.224944 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.224950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.225010 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.225068 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.227541 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.228366 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.233307 (+ 4941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.233632 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.233719 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.237137 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.237142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.237145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.237172 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.238950 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.239432 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.239437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.239441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.239523 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.239846 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.239852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.239855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.240095 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.240177 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.240744 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.240861 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260811 07:39:14.241667 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.222697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.18.72.195:58563)
0811 07:39:14.223882 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.223919 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.224699 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.225427 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.225434 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.227297 (+ 1863us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:14.227415 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.227421 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.229339 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.229342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.232506 (+ 3164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.232519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.234379 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.234386 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.234508 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.235384 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:14.235402 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:14.236957 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:14.239067 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.239070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.239072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.239322 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.239637 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.239640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.239641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.239722 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.240485 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:14.240488 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:14.240565 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.241129 (+ 564us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.241276 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1093,"mutex_wait_us":883,"spinlock_wait_cycles":9984,"thread_start_us":123,"threads_started":1}
I20260811 07:39:14.241963 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.222697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.195:43557)
0811 07:39:14.223364 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.223419 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.224690 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.225900 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.225904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.226154 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:14.226862 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.226882 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.227998 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.228002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.229387 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.229399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.230115 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.230119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.231572 (+ 1453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.232713 (+ 1141us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:14.232734 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:14.234815 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:14.237660 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.237669 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.237688 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.238059 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.238385 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.238388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.238390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.238560 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.239061 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:14.239069 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:14.239360 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.240160 (+ 800us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.240483 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":669,"thread_start_us":124,"threads_started":1}
I20260811 07:39:14.242221 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.222696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.18.72.195:43803)
0811 07:39:14.224119 (+ 1423us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.224134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.224892 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.225428 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.225436 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.227047 (+ 1611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:14.227182 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.227188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.229980 (+ 2792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.229985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.231609 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.233102 (+ 1493us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.233802 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.233808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.233928 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.234724 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:14.234744 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:14.236370 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:14.238716 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.238722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.238724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.238966 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.239272 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:14.239276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:14.239278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:14.239380 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:14.240236 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:14.240239 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:14.240349 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.241429 (+ 1080us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.241554 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"mutex_wait_us":918,"thread_start_us":106,"threads_started":1}
I20260811 07:39:14.242810 26836 heartbeater.cc:347] Connected to a master server at 127.18.72.253:46723
I20260811 07:39:14.242951 26834 heartbeater.cc:347] Connected to a master server at 127.18.72.252:35873
I20260811 07:39:14.243052 26836 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:14.243651 26836 heartbeater.cc:511] Master 127.18.72.253:46723 requested a full tablet report, sending...
I20260811 07:39:14.244243 26835 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45797
I20260811 07:39:14.244324 26835 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:14.244474 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.224545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36792 (local address 127.18.72.254:45797)
0811 07:39:14.226123 (+ 1578us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.226128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.226485 (+ 357us) server_negotiation.cc:407] Connection header received
0811 07:39:14.226535 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.226541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.226590 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.226674 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.227567 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.228325 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.230109 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.230617 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.230710 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.232731 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.232736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.232739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.232768 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.240869 (+ 8101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.243142 (+ 2273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.243146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.243149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.243241 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.243570 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.243574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.243576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.243794 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.244012 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.244165 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.244301 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1504,"thread_start_us":122,"threads_started":1}
I20260811 07:39:14.244489 26835 heartbeater.cc:511] Master 127.18.72.254:45797 requested a full tablet report, sending...
I20260811 07:39:14.243055 26834 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:14.244889 26121 ts_manager.cc:194] Registered new tserver with Master: 25c89f1b811e4b08a8e691bf1fbce46a (127.18.72.195:33449)
I20260811 07:39:14.245000 26834 heartbeater.cc:511] Master 127.18.72.252:35873 requested a full tablet report, sending...
I20260811 07:39:14.245383 25451 ts_manager.cc:194] Re-registered known tserver with Master: 25c89f1b811e4b08a8e691bf1fbce46a (127.18.72.195:33449)
I20260811 07:39:14.245471 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.224547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39128 (local address 127.18.72.252:35873)
0811 07:39:14.225474 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.225479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.225490 (+ 11us) server_negotiation.cc:407] Connection header received
0811 07:39:14.225527 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.225531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.225580 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.225662 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.226690 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.227493 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.228699 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.229045 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.229136 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.231002 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.231006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.231009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.231036 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.241589 (+ 10553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.242913 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.242918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.242920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.243014 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.243339 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.243343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.243345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.243580 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.243804 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.243997 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.244126 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":94,"threads_started":1}
I20260811 07:39:14.246032 25451 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:43557
I20260811 07:39:14.246985 26342 ts_manager.cc:194] Registered new tserver with Master: 25c89f1b811e4b08a8e691bf1fbce46a (127.18.72.195:33449)
I20260811 07:39:14.247813 25451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:39:14.250945 25451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:14.253681 26855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: NON_VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: true } } }
I20260811 07:39:14.253827 26855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:14.257004 26854 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: attempting to promote NON_VOTER 23d3225852b24df2ab3740deb6e1d15b to VOTER
I20260811 07:39:14.257330 26854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } }
I20260811 07:39:14.258409 26351 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 LEARNER]: Refusing update from remote peer 7461a49810784a64b6c0e21a9dd398e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 07:39:14.258682 26140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Refusing update from remote peer 7461a49810784a64b6c0e21a9dd398e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 07:39:14.258886 26693 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 07:39:14.260140 26693 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 07:39:14.260787 26855 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 23d3225852b24df2ab3740deb6e1d15b (127.18.72.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:14.261335 26351 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 23d3225852b24df2ab3740deb6e1d15b (127.18.72.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:14.262408 26855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:14.262507 26855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:14.262485 26693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:14.262639 26693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:14.263446 26386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:14.263553 26386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:39:14.264679 26693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:14.264801 26693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:14.265097 26140 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 23d3225852b24df2ab3740deb6e1d15b (127.18.72.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:14.267735 26863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7461a49810784a64b6c0e21a9dd398e6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:14.267876 26863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:39:14.273562 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.267504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:33449 (local address 127.0.0.1:42534)
0811 07:39:14.267767 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.267781 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.267861 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.269267 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.269270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.269290 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:14.269384 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.269391 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.270747 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.270754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.271827 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.271852 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.273126 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.273130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.273222 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.273252 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.273307 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.273402 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":74,"thread_start_us":123,"threads_started":1}
I20260811 07:39:14.273905 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.267848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:33509 (local address 127.0.0.1:37540)
0811 07:39:14.268094 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.268108 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.268251 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.269613 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.269616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.269634 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:14.269737 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.269745 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.270668 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.270675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.272344 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.272355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.273453 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.273457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.273563 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.273596 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.273654 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.273734 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":409,"thread_start_us":125,"threads_started":1}
I20260811 07:39:14.274118 26869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.268489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37540 (local address 127.18.72.193:33509)
0811 07:39:14.269080 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.269155 (+ 75us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.269182 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:39:14.269274 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.269279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.269480 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.269627 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:14.269832 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.270549 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.272457 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.273332 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.273482 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.273637 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.273711 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.273821 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":112,"threads_started":1}
I20260811 07:39:14.274124 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.266548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:45417 (local address 127.0.0.1:42772)
0811 07:39:14.266704 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.266719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.266803 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.268820 (+ 2017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.268824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.268855 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:14.268947 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.268953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.270748 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.270751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.272658 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.272667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.273606 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.273610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.273724 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.273743 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.273790 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.273848 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":619}
I20260811 07:39:14.274278 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.268492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.18.72.195:33449)
0811 07:39:14.268774 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.268788 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.268807 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:39:14.268876 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.268881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.269090 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.269287 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:14.269494 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.270604 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.272067 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.272979 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.273845 (+ 866us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.273981 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.274031 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.274119 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
I20260811 07:39:14.274808 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.267006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.18.72.194:45417)
0811 07:39:14.267401 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.267415 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.267456 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:39:14.267548 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.267556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.267804 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.268075 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:14.269398 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.270604 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.272806 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.273475 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.274391 (+ 916us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.274547 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.274598 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.274683 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":136,"threads_started":1}
I20260811 07:39:14.277449 26610 tablet_service.cc:1467] Processing CreateTablet for tablet df5e1d3da8ba449a93bd948248bf4730 (DEFAULT_TABLE table=test-table [id=f63cf0e558bb4300aece0b81ea5f636a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:14.277457 26761 tablet_service.cc:1467] Processing CreateTablet for tablet df5e1d3da8ba449a93bd948248bf4730 (DEFAULT_TABLE table=test-table [id=f63cf0e558bb4300aece0b81ea5f636a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:14.277362 26464 tablet_service.cc:1467] Processing CreateTablet for tablet df5e1d3da8ba449a93bd948248bf4730 (DEFAULT_TABLE table=test-table [id=f63cf0e558bb4300aece0b81ea5f636a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:14.278436 26610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5e1d3da8ba449a93bd948248bf4730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:14.278452 26761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5e1d3da8ba449a93bd948248bf4730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:14.278452 26464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5e1d3da8ba449a93bd948248bf4730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:14.285697 26870 tablet_bootstrap.cc:492] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Bootstrap starting.
I20260811 07:39:14.286877 26872 tablet_bootstrap.cc:492] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Bootstrap starting.
I20260811 07:39:14.288040 26870 tablet_bootstrap.cc:654] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:14.288275 26871 tablet_bootstrap.cc:492] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Bootstrap starting.
I20260811 07:39:14.288832 26870 log.cc:824] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:14.288901 26872 tablet_bootstrap.cc:654] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:14.289790 26872 log.cc:824] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:14.290545 26871 tablet_bootstrap.cc:654] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:14.290972 26870 tablet_bootstrap.cc:492] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: No bootstrap required, opened a new log
I20260811 07:39:14.291215 26870 ts_tablet_manager.cc:1397] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 07:39:14.291431 26871 log.cc:824] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:14.291944 26872 tablet_bootstrap.cc:492] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: No bootstrap required, opened a new log
I20260811 07:39:14.292156 26872 ts_tablet_manager.cc:1397] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 07:39:14.293588 26871 tablet_bootstrap.cc:492] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: No bootstrap required, opened a new log
I20260811 07:39:14.293794 26871 ts_tablet_manager.cc:1397] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:39:14.295945 26870 raft_consensus.cc:348] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:14.296209 26870 raft_consensus.cc:374] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:14.296265 26870 raft_consensus.cc:729] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3386466d6de4fb29244bcf1adc944d6, State: Initialized, Role: FOLLOWER
I20260811 07:39:14.296108 26872 raft_consensus.cc:348] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:14.296355 26872 raft_consensus.cc:374] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:14.296422 26872 raft_consensus.cc:729] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c89f1b811e4b08a8e691bf1fbce46a, State: Initialized, Role: FOLLOWER
I20260811 07:39:14.296756 26870 consensus_queue.cc:260] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:14.296857 26872 consensus_queue.cc:260] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:14.297737 26870 ts_tablet_manager.cc:1428] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:39:14.297767 26872 ts_tablet_manager.cc:1428] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:39:14.298121 26682 heartbeater.cc:503] Master 127.18.72.254:45797 was elected leader, sending a full tablet report...
I20260811 07:39:14.298062 26871 raft_consensus.cc:348] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:14.298288 26871 raft_consensus.cc:374] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:14.298344 26871 raft_consensus.cc:729] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08870894ac9541618bd48a2fe3dae028, State: Initialized, Role: FOLLOWER
I20260811 07:39:14.298708 26871 consensus_queue.cc:260] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:14.299204 26835 heartbeater.cc:503] Master 127.18.72.254:45797 was elected leader, sending a full tablet report...
I20260811 07:39:14.302057 26871 ts_tablet_manager.cc:1428] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 07:39:14.302241 26534 heartbeater.cc:503] Master 127.18.72.254:45797 was elected leader, sending a full tablet report...
W20260811 07:39:14.471602 26837 tablet.cc:2369] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:39:14.508160 26685 tablet.cc:2369] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:14.510198 26877 raft_consensus.cc:482] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:14.510421 26877 raft_consensus.cc:504] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:14.511626 26877 leader_election.cc:288] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3386466d6de4fb29244bcf1adc944d6 (127.18.72.194:45417), 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509)
I20260811 07:39:14.515476 26869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.512046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:43943 (local address 127.18.72.193:33509)
0811 07:39:14.512176 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.512182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.512244 (+ 62us) server_negotiation.cc:407] Connection header received
0811 07:39:14.512426 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.512429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.512478 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.512562 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:14.512868 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.513379 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.514509 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.514895 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.515021 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.515078 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.515263 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.515328 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:39:14.515467 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.511965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:33509 (local address 127.18.72.195:43943)
0811 07:39:14.512174 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.512189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.512309 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.512576 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.512578 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.512601 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:14.512745 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.512751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.513493 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.513498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.514381 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.514391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.515016 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.515019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.515133 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.515155 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.515233 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.515308 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":103,"spinlock_wait_cycles":8192}
I20260811 07:39:14.515564 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.511946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:45417 (local address 127.18.72.195:35473)
0811 07:39:14.512176 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:14.512193 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:14.512301 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:14.512582 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:14.512585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:14.512607 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:14.512754 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.512760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.513480 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.513486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.514390 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:14.514399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.515151 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.515154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:14.515269 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.515295 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:39:14.515363 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 07:39:14.515418 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":81,"spinlock_wait_cycles":6656}
I20260811 07:39:14.515587 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.512035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:35473 (local address 127.18.72.194:45417)
0811 07:39:14.512240 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.512244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.512260 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:39:14.512443 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.512446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.512490 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.512571 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:14.512875 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.513369 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.514520 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.514979 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.515095 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.515157 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.515398 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.515469 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 07:39:14.516181 26485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5e1d3da8ba449a93bd948248bf4730" candidate_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08870894ac9541618bd48a2fe3dae028" is_pre_election: true
I20260811 07:39:14.516188 26633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5e1d3da8ba449a93bd948248bf4730" candidate_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3386466d6de4fb29244bcf1adc944d6" is_pre_election: true
I20260811 07:39:14.516487 26485 raft_consensus.cc:2454] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25c89f1b811e4b08a8e691bf1fbce46a in term 0.
I20260811 07:39:14.516503 26633 raft_consensus.cc:2454] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25c89f1b811e4b08a8e691bf1fbce46a in term 0.
I20260811 07:39:14.516955 26716 leader_election.cc:302] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c89f1b811e4b08a8e691bf1fbce46a, d3386466d6de4fb29244bcf1adc944d6; no voters:
I20260811 07:39:14.517251 26877 raft_consensus.cc:2781] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:14.517354 26877 raft_consensus.cc:482] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:14.517401 26877 raft_consensus.cc:3037] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:14.518431 26877 raft_consensus.cc:504] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:14.518765 26877 leader_election.cc:288] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [CANDIDATE]: Term 1 election: Requested vote from peers d3386466d6de4fb29244bcf1adc944d6 (127.18.72.194:45417), 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509)
I20260811 07:39:14.519137 26485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5e1d3da8ba449a93bd948248bf4730" candidate_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08870894ac9541618bd48a2fe3dae028"
I20260811 07:39:14.519124 26633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5e1d3da8ba449a93bd948248bf4730" candidate_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3386466d6de4fb29244bcf1adc944d6"
I20260811 07:39:14.519256 26633 raft_consensus.cc:3037] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:14.519265 26485 raft_consensus.cc:3037] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:14.520308 26633 raft_consensus.cc:2454] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25c89f1b811e4b08a8e691bf1fbce46a in term 1.
I20260811 07:39:14.520465 26485 raft_consensus.cc:2454] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25c89f1b811e4b08a8e691bf1fbce46a in term 1.
I20260811 07:39:14.520573 26716 leader_election.cc:302] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c89f1b811e4b08a8e691bf1fbce46a, d3386466d6de4fb29244bcf1adc944d6; no voters:
I20260811 07:39:14.520773 26877 raft_consensus.cc:2781] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:14.521016 26877 raft_consensus.cc:686] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 LEADER]: Becoming Leader. State: Replica: 25c89f1b811e4b08a8e691bf1fbce46a, State: Running, Role: LEADER
I20260811 07:39:14.521430 26877 consensus_queue.cc:237] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
W20260811 07:39:14.522253 26536 tablet.cc:2369] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:14.525481 25448 catalog_manager.cc:5495] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a reported cstate change: term changed from 0 to 1, leader changed from <none> to 25c89f1b811e4b08a8e691bf1fbce46a (127.18.72.195). New cstate: current_term: 1 leader_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } health_report { overall_health: UNKNOWN } } }
I20260811 07:39:14.565390 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.561781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.18.72.195:33449)
0811 07:39:14.561927 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.561934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.561951 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:39:14.562048 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.562050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.562105 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.562223 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:14.562472 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.562999 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.563869 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.564106 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.564221 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.564540 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:14.564981 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:14.565121 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.565158 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.565219 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:39:14.571877 26485 raft_consensus.cc:1261] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 1 FOLLOWER]: Refusing update from remote peer 25c89f1b811e4b08a8e691bf1fbce46a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:14.571884 26633 raft_consensus.cc:1261] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 1 FOLLOWER]: Refusing update from remote peer 25c89f1b811e4b08a8e691bf1fbce46a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:14.572757 26879 consensus_queue.cc:1035] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:14.573027 26877 consensus_queue.cc:1035] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [LEADER]: Connected to new peer: Peer: permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:14.582662 26884 mvcc.cc:205] Tried to move back new op lower bound from 7317233477918683136 to 7317233477722718208. Current Snapshot: MvccSnapshot[applied={T|T < 7317233477918683136}]
I20260811 07:39:14.586359 26883 mvcc.cc:205] Tried to move back new op lower bound from 7317233477918683136 to 7317233477722718208. Current Snapshot: MvccSnapshot[applied={T|T < 7317233477918683136}]
I20260811 07:39:14.591967 25449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:14.592118 25449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:14.596971 25449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7461a49810784a64b6c0e21a9dd398e6: Sending DeleteTablet for 3 replicas of tablet df5e1d3da8ba449a93bd948248bf4730
I20260811 07:39:14.597620 26610 tablet_service.cc:1514] Processing DeleteTablet for tablet df5e1d3da8ba449a93bd948248bf4730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:14 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:42772
I20260811 07:39:14.597839 26464 tablet_service.cc:1514] Processing DeleteTablet for tablet df5e1d3da8ba449a93bd948248bf4730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:14 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:37540
I20260811 07:39:14.597971 26761 tablet_service.cc:1514] Processing DeleteTablet for tablet df5e1d3da8ba449a93bd948248bf4730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:14 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:42534
I20260811 07:39:14.598405 26892 tablet_replica.cc:330] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: stopping tablet replica
I20260811 07:39:14.598769 26892 raft_consensus.cc:2229] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:14.600014 26892 raft_consensus.cc:2258] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:14.600045 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 25419
I20260811 07:39:14.604005 26892 ts_tablet_manager.cc:1905] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:14.606974 26893 tablet_replica.cc:330] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: stopping tablet replica
I20260811 07:39:14.607302 26892 ts_tablet_manager.cc:1918] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:14.607319 26893 raft_consensus.cc:2229] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:14.607400 26892 log.cc:1196] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/df5e1d3da8ba449a93bd948248bf4730
I20260811 07:39:14.607645 26893 raft_consensus.cc:2258] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:14.607741 26892 ts_tablet_manager.cc:1939] T df5e1d3da8ba449a93bd948248bf4730 P d3386466d6de4fb29244bcf1adc944d6: Deleting consensus metadata
I20260811 07:39:14.608129 26894 tablet_replica.cc:330] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: stopping tablet replica
I20260811 07:39:14.608438 26894 raft_consensus.cc:2229] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:39:14.608858 26894 raft_consensus.cc:2258] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:14.609805 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.599154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.18.72.253:46723)
0811 07:39:14.599291 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.599296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.599308 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:14.599356 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.599359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.599403 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.599485 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.601207 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.602029 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.603222 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.603523 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.603602 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.606053 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.606056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.606059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.606088 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.607913 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.608422 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.608426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.608427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.608526 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.608844 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.608847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.608848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.609079 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.609196 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.609524 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.609638 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 07:39:14.613224 26562 connection.cc:535] server connection from 127.0.0.1:42772 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 07:39:14.613571 26893 ts_tablet_manager.cc:1905] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:14.616823 26893 ts_tablet_manager.cc:1918] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:14.616936 26893 log.cc:1196] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/df5e1d3da8ba449a93bd948248bf4730
I20260811 07:39:14.617307 26893 ts_tablet_manager.cc:1939] T df5e1d3da8ba449a93bd948248bf4730 P 08870894ac9541618bd48a2fe3dae028: Deleting consensus metadata
W20260811 07:39:14.618592 26412 connection.cc:441] server connection from 127.0.0.1:37540 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37540 (request call id 1) could send its response
I20260811 07:39:14.620425 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.598935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39134 (local address 127.18.72.252:35873)
0811 07:39:14.599052 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.599057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.609885 (+ 10828us) server_negotiation.cc:407] Connection header received
0811 07:39:14.610104 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.610107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.610164 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.610240 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.611072 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.611592 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.612656 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.612998 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.613077 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.615164 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.615168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.615171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.615200 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.617035 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.619050 (+ 2015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.619054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.619056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.619142 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.619435 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.619438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.619440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.619626 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.619725 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.620149 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.620271 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 07:39:14.621335 26894 ts_tablet_manager.cc:1905] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:14.624576 26894 ts_tablet_manager.cc:1918] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:14.624706 26894 log.cc:1196] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/df5e1d3da8ba449a93bd948248bf4730
I20260811 07:39:14.625123 26894 ts_tablet_manager.cc:1939] T df5e1d3da8ba449a93bd948248bf4730 P 25c89f1b811e4b08a8e691bf1fbce46a: Deleting consensus metadata
W20260811 07:39:14.626317 26713 connection.cc:441] server connection from 127.0.0.1:42534 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42534 (request call id 1) could send its response
I20260811 07:39:14.626830 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.617362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39136 (local address 127.18.72.252:35873)
0811 07:39:14.617483 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.617490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.617695 (+ 205us) server_negotiation.cc:407] Connection header received
0811 07:39:14.617827 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.617830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.617875 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.617947 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.618837 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.619596 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.620975 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.621250 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.621338 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.623141 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.623145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.623147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.623170 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.625092 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.625568 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.625571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.625573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.625655 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.625929 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.625933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.625935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.626104 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.626244 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.626534 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.626643 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:39:14.627293 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:14.617104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.18.72.253:46723)
0811 07:39:14.617271 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:14.617276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:14.617968 (+ 692us) server_negotiation.cc:407] Connection header received
0811 07:39:14.618116 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:14.618119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:14.618159 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:14.618206 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:14.619278 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.620058 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.621220 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:14.621532 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:14.621607 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:14.623451 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:14.623455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:14.623459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:14.623486 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:14.625443 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.625937 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.625941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.625943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.626032 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:14.626370 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:14.626374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:14.626376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:14.626589 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:14.626699 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:14.627038 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 07:39:14.627152 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 07:39:15.585246 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:15.584684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.195:36777)
0811 07:39:15.585011 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:15.585123 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":79,"threads_started":1}
W20260811 07:39:15.585562 26835 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:45797 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
I20260811 07:39:15.586572 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:15.586180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.194:38377)
0811 07:39:15.586411 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:15.586495 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
W20260811 07:39:15.586899 26682 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:45797 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
I20260811 07:39:15.588979 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:15.588617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.193:58881)
0811 07:39:15.588834 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:15.588902 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":59,"threads_started":1}
W20260811 07:39:15.589236 26534 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:45797 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
I20260811 07:39:16.096562 26906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7461a49810784a64b6c0e21a9dd398e6)
I20260811 07:39:16.096818 26906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } }
I20260811 07:39:16.097534 26907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7461a49810784a64b6c0e21a9dd398e6)
I20260811 07:39:16.097702 26907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } }
I20260811 07:39:16.098083 26906 leader_election.cc:288] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7461a49810784a64b6c0e21a9dd398e6 (127.18.72.254:45797), 6251da0ffb45473fb96b1fb0abd9d14b (127.18.72.253:46723)
I20260811 07:39:16.098702 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.098168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:37304)
0811 07:39:16.098503 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.098586 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":57,"threads_started":1}
W20260811 07:39:16.098995 26324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
I20260811 07:39:16.099265 26907 leader_election.cc:288] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7461a49810784a64b6c0e21a9dd398e6 (127.18.72.254:45797), 23d3225852b24df2ab3740deb6e1d15b (127.18.72.252:35873)
I20260811 07:39:16.099321 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.098819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:37316)
0811 07:39:16.099156 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.099234 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":81,"threads_started":1}
W20260811 07:39:16.099493 26112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
I20260811 07:39:16.099924 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.099699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:37318)
0811 07:39:16.099786 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.099837 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 07:39:16.100077 26324 leader_election.cc:334] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7461a49810784a64b6c0e21a9dd398e6 (127.18.72.254:45797): Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
I20260811 07:39:16.100878 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.100602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:37334)
0811 07:39:16.100755 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.100804 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260811 07:39:16.100975 26112 leader_election.cc:334] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7461a49810784a64b6c0e21a9dd398e6 (127.18.72.254:45797): Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
I20260811 07:39:16.102732 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.098510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:46723 (local address 127.0.0.1:60492)
0811 07:39:16.098797 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.098813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:16.098935 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:16.099368 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:16.099371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:16.099393 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:16.099500 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.099507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.100574 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.100578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.101659 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.101670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.102331 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.102335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.102443 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.102469 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:39:16.102547 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:39:16.102615 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":63,"threads_started":1}
I20260811 07:39:16.102816 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.098353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.18.72.253:46723)
0811 07:39:16.098583 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:16.098588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:16.099034 (+ 446us) server_negotiation.cc:407] Connection header received
0811 07:39:16.099098 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:16.099101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:16.099153 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:16.099233 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:16.099633 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.100448 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.101784 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.102212 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.102293 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.102343 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:16.102592 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 07:39:16.102684 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":59,"threads_started":1}
I20260811 07:39:16.103240 26140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23d3225852b24df2ab3740deb6e1d15b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" is_pre_election: true
I20260811 07:39:16.103453 26140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d3225852b24df2ab3740deb6e1d15b in term 1.
I20260811 07:39:16.103858 26325 leader_election.cc:302] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23d3225852b24df2ab3740deb6e1d15b, 6251da0ffb45473fb96b1fb0abd9d14b; no voters: 7461a49810784a64b6c0e21a9dd398e6
I20260811 07:39:16.104111 26906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 07:39:16.104184 26906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7461a49810784a64b6c0e21a9dd398e6)
I20260811 07:39:16.104213 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.098929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:35873 (local address 127.0.0.1:39152)
0811 07:39:16.099908 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.099922 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:16.100011 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:16.100448 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:16.100452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:16.100475 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:16.100570 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.100575 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.101492 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.101496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.103053 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.103085 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.103803 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.103806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.103918 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.103945 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 07:39:16.104023 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:39:16.104093 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"thread_start_us":75,"threads_started":1}
I20260811 07:39:16.104238 26906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:39:16.104610 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.099354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39152 (local address 127.18.72.252:35873)
0811 07:39:16.100223 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:16.100232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:16.100245 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:16.100289 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:16.100291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:16.100335 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:16.100425 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:16.100681 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.101372 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.103180 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.103689 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.103791 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.103851 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:16.104465 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 07:39:16.104525 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":92,"threads_started":1}
I20260811 07:39:16.105006 26351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "23d3225852b24df2ab3740deb6e1d15b" is_pre_election: true
I20260811 07:39:16.105576 26906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } }
I20260811 07:39:16.105815 26351 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6251da0ffb45473fb96b1fb0abd9d14b in current term 2: Already voted for candidate 23d3225852b24df2ab3740deb6e1d15b in this term.
I20260811 07:39:16.105974 26906 leader_election.cc:288] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [CANDIDATE]: Term 2 election: Requested vote from peers 7461a49810784a64b6c0e21a9dd398e6 (127.18.72.254:45797), 6251da0ffb45473fb96b1fb0abd9d14b (127.18.72.253:46723)
I20260811 07:39:16.106302 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.106076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:37344)
0811 07:39:16.106192 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.106246 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 07:39:16.106320 26140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23d3225852b24df2ab3740deb6e1d15b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6251da0ffb45473fb96b1fb0abd9d14b"
I20260811 07:39:16.106383 26115 leader_election.cc:302] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6251da0ffb45473fb96b1fb0abd9d14b; no voters: 23d3225852b24df2ab3740deb6e1d15b, 7461a49810784a64b6c0e21a9dd398e6
I20260811 07:39:16.106426 26140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:39:16.107398 26140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d3225852b24df2ab3740deb6e1d15b in term 2.
I20260811 07:39:16.107492 26907 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:39:16.107688 26325 leader_election.cc:302] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d3225852b24df2ab3740deb6e1d15b, 6251da0ffb45473fb96b1fb0abd9d14b; no voters:
I20260811 07:39:16.107864 26906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:39:16.108109 26906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [term 2 LEADER]: Becoming Leader. State: Replica: 23d3225852b24df2ab3740deb6e1d15b, State: Running, Role: LEADER
I20260811 07:39:16.108349 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.108089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:37350)
0811 07:39:16.108229 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.108285 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 07:39:16.108453 26324 leader_election.cc:334] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7461a49810784a64b6c0e21a9dd398e6 (127.18.72.254:45797): Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
I20260811 07:39:16.108403 26906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } }
I20260811 07:39:16.109743 26917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23d3225852b24df2ab3740deb6e1d15b. Latest consensus state: current_term: 2 leader_uuid: "23d3225852b24df2ab3740deb6e1d15b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:16.109839 26917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:16.110072 26919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:16.112238 26919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:16.112627 26919 catalog_manager.cc:1259] Loaded cluster ID: e73456e2e8d0444db1bd5ea14054a285
I20260811 07:39:16.112696 26919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:16.113242 26919 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:16.113629 26919 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Loaded TSK: 0
I20260811 07:39:16.114382 26919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:39:16.147028 26339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:39:16.150300 26339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:16.153324 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.153080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:37354)
0811 07:39:16.153199 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.153257 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 07:39:16.153486 26140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [term 2 FOLLOWER]: Refusing update from remote peer 23d3225852b24df2ab3740deb6e1d15b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 07:39:16.153960 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.153752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.0.0.1:37358)
0811 07:39:16.153861 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.153911 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 07:39:16.154071 26324 consensus_peers.cc:487] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b -> Peer 7461a49810784a64b6c0e21a9dd398e6 (127.18.72.254:45797): Couldn't send request to peer 7461a49810784a64b6c0e21a9dd398e6. Status: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 07:39:16.154112 26917 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 07:39:16.155578 26907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23d3225852b24df2ab3740deb6e1d15b. Latest consensus state: current_term: 2 leader_uuid: "23d3225852b24df2ab3740deb6e1d15b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:16.155685 26907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:39:16.157007 26917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "23d3225852b24df2ab3740deb6e1d15b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:16.157128 26917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:16.157357 26907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "23d3225852b24df2ab3740deb6e1d15b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7461a49810784a64b6c0e21a9dd398e6" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45797 } } peers { permanent_uuid: "6251da0ffb45473fb96b1fb0abd9d14b" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 46723 } attrs { promote: false } } peers { permanent_uuid: "23d3225852b24df2ab3740deb6e1d15b" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 35873 } attrs { promote: false } } }
I20260811 07:39:16.157490 26907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6251da0ffb45473fb96b1fb0abd9d14b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:39:16.157615 26921 mvcc.cc:205] Tried to move back new op lower bound from 7317233484399521792 to 7317233484222472192. Current Snapshot: MvccSnapshot[applied={T|T < 7317233484399521792}]
I20260811 07:39:16.167867 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.163767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:33509 (local address 127.0.0.1:37554)
0811 07:39:16.164427 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.164441 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:16.164535 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:16.164762 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:16.164765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:16.164782 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:16.164873 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.164879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.165548 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.165553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.166579 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.166589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.167439 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.167442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.167528 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.167559 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 07:39:16.167648 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 07:39:16.167716 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":46,"thread_start_us":125,"threads_started":1}
I20260811 07:39:16.167866 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.164157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.18.72.193:33509)
0811 07:39:16.164405 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:16.164409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:16.164495 (+ 86us) server_negotiation.cc:407] Connection header received
0811 07:39:16.164622 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:16.164624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:16.164662 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:16.164726 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:16.164954 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.165436 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.166738 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.167288 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.167432 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.167500 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:16.167664 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 07:39:16.167740 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":89,"threads_started":1}
I20260811 07:39:16.168170 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.163194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42788 (local address 127.18.72.194:45417)
0811 07:39:16.163713 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:16.163718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:16.163732 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:16.163788 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:16.163793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:16.163852 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:16.164343 (+ 491us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:16.164475 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.165208 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.166657 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.167327 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.167470 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.167532 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:16.167944 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 07:39:16.168040 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":88,"threads_started":1}
I20260811 07:39:16.168577 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.163068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:45417 (local address 127.0.0.1:42788)
0811 07:39:16.163163 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.163176 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:16.163254 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:16.164016 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:16.164020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:16.164039 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:16.164152 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.164158 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.165313 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.165319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.166519 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.166529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.167614 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.167617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.167700 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.167723 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 07:39:16.167783 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 07:39:16.167859 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":25}
I20260811 07:39:16.168812 26464 tablet_service.cc:1467] Processing CreateTablet for tablet bfae5776b69b430c8884ee491503a6da (DEFAULT_TABLE table=test-table [id=9e5ca8b897fd4942b7ada00578170d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:16.169126 26464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfae5776b69b430c8884ee491503a6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:16.169181 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.163524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:33449 (local address 127.0.0.1:42552)
0811 07:39:16.163993 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.164007 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:16.164102 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:16.165689 (+ 1587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:16.165692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:16.165712 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:16.165793 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.165799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.166789 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.166792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.167867 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.167877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.168844 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.168848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.168937 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.168963 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:39:16.169015 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 07:39:16.169079 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404}
I20260811 07:39:16.169158 26610 tablet_service.cc:1467] Processing CreateTablet for tablet bfae5776b69b430c8884ee491503a6da (DEFAULT_TABLE table=test-table [id=9e5ca8b897fd4942b7ada00578170d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:16.169446 26610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfae5776b69b430c8884ee491503a6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:16.169775 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.163800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.18.72.195:33449)
0811 07:39:16.165467 (+ 1667us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:16.165472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:16.165485 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:16.165535 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:16.165538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:16.165583 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:16.165673 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:16.165918 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.166672 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.168064 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.168702 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.169446 (+ 744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.169517 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:16.169555 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 07:39:16.169627 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1585,"thread_start_us":414,"threads_started":1}
I20260811 07:39:16.170208 26761 tablet_service.cc:1467] Processing CreateTablet for tablet bfae5776b69b430c8884ee491503a6da (DEFAULT_TABLE table=test-table [id=9e5ca8b897fd4942b7ada00578170d38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:16.170473 26761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfae5776b69b430c8884ee491503a6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:16.171826 26930 tablet_bootstrap.cc:492] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028: Bootstrap starting.
I20260811 07:39:16.172206 26931 tablet_bootstrap.cc:492] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: Bootstrap starting.
I20260811 07:39:16.172791 26930 tablet_bootstrap.cc:654] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:16.173108 26932 tablet_bootstrap.cc:492] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: Bootstrap starting.
I20260811 07:39:16.173163 26931 tablet_bootstrap.cc:654] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:16.174053 26932 tablet_bootstrap.cc:654] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:16.174197 26930 tablet_bootstrap.cc:492] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028: No bootstrap required, opened a new log
I20260811 07:39:16.174317 26930 ts_tablet_manager.cc:1397] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 07:39:16.174420 26931 tablet_bootstrap.cc:492] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: No bootstrap required, opened a new log
I20260811 07:39:16.174518 26931 ts_tablet_manager.cc:1397] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:16.174801 26930 raft_consensus.cc:348] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.174937 26930 raft_consensus.cc:374] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:16.174975 26930 raft_consensus.cc:729] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08870894ac9541618bd48a2fe3dae028, State: Initialized, Role: FOLLOWER
I20260811 07:39:16.175004 26931 raft_consensus.cc:348] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.175119 26931 raft_consensus.cc:374] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:16.175158 26932 tablet_bootstrap.cc:492] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: No bootstrap required, opened a new log
I20260811 07:39:16.175168 26931 raft_consensus.cc:729] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3386466d6de4fb29244bcf1adc944d6, State: Initialized, Role: FOLLOWER
I20260811 07:39:16.175122 26930 consensus_queue.cc:260] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.175244 26932 ts_tablet_manager.cc:1397] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:16.175318 26931 consensus_queue.cc:260] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.175486 26930 ts_tablet_manager.cc:1428] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 07:39:16.175642 26931 ts_tablet_manager.cc:1428] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 07:39:16.175658 26932 raft_consensus.cc:348] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.175761 26932 raft_consensus.cc:374] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:16.175791 26932 raft_consensus.cc:729] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c89f1b811e4b08a8e691bf1fbce46a, State: Initialized, Role: FOLLOWER
I20260811 07:39:16.175921 26932 consensus_queue.cc:260] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.176245 26932 ts_tablet_manager.cc:1428] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:16.178086 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.176967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.195:59617)
0811 07:39:16.177479 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.177539 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":90,"threads_started":1}
I20260811 07:39:16.178743 26533 heartbeater.cc:503] Master 127.18.72.252:35873 was elected leader, sending a full tablet report...
I20260811 07:39:16.178839 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.178508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.195:47577)
0811 07:39:16.178727 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.178778 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260811 07:39:16.179353 26834 heartbeater.cc:503] Master 127.18.72.252:35873 was elected leader, sending a full tablet report...
I20260811 07:39:16.180397 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.176502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.193:46397)
0811 07:39:16.178195 (+ 1693us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.178252 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1610,"thread_start_us":93,"threads_started":1}
I20260811 07:39:16.181196 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.180869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.193:54515)
0811 07:39:16.181077 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.181130 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 07:39:16.180948 26681 heartbeater.cc:503] Master 127.18.72.252:35873 was elected leader, sending a full tablet report...
I20260811 07:39:16.180862 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.179729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.194:34105)
0811 07:39:16.180702 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.180766 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":95,"threads_started":1}
I20260811 07:39:16.182267 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.181952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.194:33091)
0811 07:39:16.182152 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.182196 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260811 07:39:16.288925 26938 raft_consensus.cc:482] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:16.289076 26938 raft_consensus.cc:504] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.290360 26938 leader_election.cc:288] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25c89f1b811e4b08a8e691bf1fbce46a (127.18.72.195:33449), 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509)
I20260811 07:39:16.295861 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.290729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:47313 (local address 127.18.72.195:33449)
0811 07:39:16.290874 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:16.290880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:16.290899 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:39:16.291152 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:16.291156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:16.291215 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:16.291308 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:16.291628 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.292593 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.294454 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.295078 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.295185 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.295239 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:16.295661 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 07:39:16.295745 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:39:16.295845 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.291074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:46709 (local address 127.18.72.193:33509)
0811 07:39:16.291265 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:16.291271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:16.291289 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:39:16.291343 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:16.291346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:16.291391 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:16.291466 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:16.291852 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.292614 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.294455 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.295052 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.295184 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.295239 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:16.295627 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 07:39:16.295711 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 07:39:16.295847 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.290634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:33449 (local address 127.18.72.194:47313)
0811 07:39:16.290772 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.290790 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:16.290911 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:16.291322 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:16.291325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:16.291346 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:16.291497 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.291504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.292726 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.292729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.294318 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.294330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.295224 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.295228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.295456 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.295514 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 07:39:16.295590 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 07:39:16.295671 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":195}
I20260811 07:39:16.295847 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.290943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:33509 (local address 127.18.72.194:46709)
0811 07:39:16.291078 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.291093 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:16.291246 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:16.291496 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:16.291499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:16.291523 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:16.291648 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.291654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.292726 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.292729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.294317 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:16.294330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.295224 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.295228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:16.295462 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.295502 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 07:39:16.295635 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 07:39:16.295688 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":238}
I20260811 07:39:16.296535 26485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfae5776b69b430c8884ee491503a6da" candidate_uuid: "d3386466d6de4fb29244bcf1adc944d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08870894ac9541618bd48a2fe3dae028" is_pre_election: true
I20260811 07:39:16.296622 26783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfae5776b69b430c8884ee491503a6da" candidate_uuid: "d3386466d6de4fb29244bcf1adc944d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" is_pre_election: true
I20260811 07:39:16.296725 26485 raft_consensus.cc:2454] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3386466d6de4fb29244bcf1adc944d6 in term 0.
I20260811 07:39:16.296927 26783 raft_consensus.cc:2454] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3386466d6de4fb29244bcf1adc944d6 in term 0.
I20260811 07:39:16.297191 26561 leader_election.cc:302] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08870894ac9541618bd48a2fe3dae028, d3386466d6de4fb29244bcf1adc944d6; no voters:
I20260811 07:39:16.297462 26938 raft_consensus.cc:2781] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:16.297520 26938 raft_consensus.cc:482] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:16.297547 26938 raft_consensus.cc:3037] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:16.298617 26938 raft_consensus.cc:504] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.298965 26938 leader_election.cc:288] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [CANDIDATE]: Term 1 election: Requested vote from peers 25c89f1b811e4b08a8e691bf1fbce46a (127.18.72.195:33449), 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509)
I20260811 07:39:16.299321 26783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfae5776b69b430c8884ee491503a6da" candidate_uuid: "d3386466d6de4fb29244bcf1adc944d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c89f1b811e4b08a8e691bf1fbce46a"
I20260811 07:39:16.299423 26783 raft_consensus.cc:3037] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:16.299428 26485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfae5776b69b430c8884ee491503a6da" candidate_uuid: "d3386466d6de4fb29244bcf1adc944d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08870894ac9541618bd48a2fe3dae028"
I20260811 07:39:16.299551 26485 raft_consensus.cc:3037] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:16.300498 26485 raft_consensus.cc:2454] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3386466d6de4fb29244bcf1adc944d6 in term 1.
I20260811 07:39:16.300499 26783 raft_consensus.cc:2454] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3386466d6de4fb29244bcf1adc944d6 in term 1.
I20260811 07:39:16.300778 26561 leader_election.cc:302] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c89f1b811e4b08a8e691bf1fbce46a, d3386466d6de4fb29244bcf1adc944d6; no voters:
I20260811 07:39:16.300968 26938 raft_consensus.cc:2781] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:16.301221 26938 raft_consensus.cc:686] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 1 LEADER]: Becoming Leader. State: Replica: d3386466d6de4fb29244bcf1adc944d6, State: Running, Role: LEADER
I20260811 07:39:16.301617 26938 consensus_queue.cc:237] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } }
I20260811 07:39:16.301939 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.301657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.194:57873)
0811 07:39:16.301806 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.301874 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 07:39:16.304102 26339 catalog_manager.cc:5495] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3386466d6de4fb29244bcf1adc944d6 (127.18.72.194). New cstate: current_term: 1 leader_uuid: "d3386466d6de4fb29244bcf1adc944d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3386466d6de4fb29244bcf1adc944d6" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 } health_report { overall_health: UNKNOWN } } }
I20260811 07:39:16.320770 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.317191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.18.72.194:45417)
0811 07:39:16.317329 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:16.317334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:16.317348 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:16.317507 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:16.317509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:16.317565 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:16.317676 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:16.317908 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.318470 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.319317 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:16.319532 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:16.319637 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:16.319946 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:16.320388 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:16.320517 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:16.320581 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:39:16.320634 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:39:16.323396 26485 raft_consensus.cc:1261] T bfae5776b69b430c8884ee491503a6da P 08870894ac9541618bd48a2fe3dae028 [term 1 FOLLOWER]: Refusing update from remote peer d3386466d6de4fb29244bcf1adc944d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:16.323784 26783 raft_consensus.cc:1261] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 FOLLOWER]: Refusing update from remote peer d3386466d6de4fb29244bcf1adc944d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:16.324169 26938 consensus_queue.cc:1035] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08870894ac9541618bd48a2fe3dae028" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 33509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:16.324391 26944 consensus_queue.cc:1035] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25c89f1b811e4b08a8e691bf1fbce46a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:16.326166 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.325914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.193:53695)
0811 07:39:16.326032 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.326091 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 07:39:16.329222 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.328961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.194:57689)
0811 07:39:16.329098 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.329150 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 07:39:16.329881 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.329478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.195:45561)
0811 07:39:16.329756 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.329811 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260811 07:39:16.331959 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.331665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.195:37833)
0811 07:39:16.331794 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.331855 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 07:39:16.332446 26951 mvcc.cc:205] Tried to move back new op lower bound from 7317233485095804928 to 7317233485014175744. Current Snapshot: MvccSnapshot[applied={T|T < 7317233485095804928}]
I20260811 07:39:16.335199 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.334800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.193:38821)
0811 07:39:16.335074 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.335128 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260811 07:39:16.335392 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.335063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45797 (local address 127.18.72.194:39775)
0811 07:39:16.335258 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.335323 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 07:39:16.338227 26342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:16.338387 26342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:16.342227 26342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23d3225852b24df2ab3740deb6e1d15b: Sending DeleteTablet for 3 replicas of tablet bfae5776b69b430c8884ee491503a6da
I20260811 07:39:16.342849 26610 tablet_service.cc:1514] Processing DeleteTablet for tablet bfae5776b69b430c8884ee491503a6da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:16 UTC) from {username='oryx', principal='oryx/127.18.72.252@KRBTEST.COM'} at 127.0.0.1:42788
I20260811 07:39:16.343150 26464 tablet_service.cc:1514] Processing DeleteTablet for tablet bfae5776b69b430c8884ee491503a6da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:16 UTC) from {username='oryx', principal='oryx/127.18.72.252@KRBTEST.COM'} at 127.0.0.1:37554
I20260811 07:39:16.343528 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 26295
I20260811 07:39:16.343561 26761 tablet_service.cc:1514] Processing DeleteTablet for tablet bfae5776b69b430c8884ee491503a6da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:16 UTC) from {username='oryx', principal='oryx/127.18.72.252@KRBTEST.COM'} at 127.0.0.1:42552
I20260811 07:39:16.347224 26962 tablet_replica.cc:330] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: stopping tablet replica
I20260811 07:39:16.347384 26962 raft_consensus.cc:2229] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:16.347536 26962 raft_consensus.cc:2258] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:16.348237 26962 ts_tablet_manager.cc:1905] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:16.350142 26959 tablet_replica.cc:330] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: stopping tablet replica
I20260811 07:39:16.350272 26959 raft_consensus.cc:2229] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:39:16.350485 26962 ts_tablet_manager.cc:1918] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:16.350579 26962 log.cc:1196] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/bfae5776b69b430c8884ee491503a6da
I20260811 07:39:16.350571 26959 raft_consensus.cc:2258] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:16.350907 26962 ts_tablet_manager.cc:1939] T bfae5776b69b430c8884ee491503a6da P 25c89f1b811e4b08a8e691bf1fbce46a: Deleting consensus metadata
I20260811 07:39:16.351660 26326 catalog_manager.cc:4841] TS 25c89f1b811e4b08a8e691bf1fbce46a (127.18.72.195:33449): tablet bfae5776b69b430c8884ee491503a6da (table test-table [id=9e5ca8b897fd4942b7ada00578170d38]) successfully deleted
I20260811 07:39:16.352532 26959 ts_tablet_manager.cc:1905] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:16.354768 26959 ts_tablet_manager.cc:1918] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:16.354866 26959 log.cc:1196] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/bfae5776b69b430c8884ee491503a6da
I20260811 07:39:16.355155 26959 ts_tablet_manager.cc:1939] T bfae5776b69b430c8884ee491503a6da P d3386466d6de4fb29244bcf1adc944d6: Deleting consensus metadata
I20260811 07:39:16.355792 26327 catalog_manager.cc:4841] TS d3386466d6de4fb29244bcf1adc944d6 (127.18.72.194:45417): tablet bfae5776b69b430c8884ee491503a6da (table test-table [id=9e5ca8b897fd4942b7ada00578170d38]) successfully deleted
I20260811 07:39:16.358747 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 26541
I20260811 07:39:16.358913 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:16.358591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:33509 (local address 127.0.0.1:37568)
0811 07:39:16.358715 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:16.358824 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:33509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 07:39:16.359768 26326 catalog_manager.cc:4584] TS 08870894ac9541618bd48a2fe3dae028 (127.18.72.193:33509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bfae5776b69b430c8884ee491503a6da: Network error: Client connection negotiation failed: client connection to 127.18.72.193:33509: connect: Connection refused (error 111)
I20260811 07:39:16.370245 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 26689
I20260811 07:39:16.379083 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 25972
I20260811 07:39:16.386471 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 26091
2026-08-11T07:39:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5680 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:16 dist-test-slave-zkpd krb5kdc[26965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:16 dist-test-slave-zkpd krb5kdc[26965](info): set up 2 sockets
Aug 11 07:39:16 dist-test-slave-zkpd krb5kdc[26965](info): commencing operation
krb5kdc: starting...
W20260811 07:39:17.493438 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:17 dist-test-slave-zkpd krb5kdc[26965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.254.keytab.
Entry for principal oryx/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.254.keytab.
I20260811 07:39:17.618263 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:33387
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.254.keytab
--principal=oryx/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:33387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:17.729060 26979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:17.729295 26979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:17.729333 26979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:17.733263 26979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:17.733309 26979 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 07:39:17.733325 26979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:17.733341 26979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:17.733404 26979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:17.733431 26979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:17.738508 26979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:33387
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.254.keytab
--principal=oryx/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:33387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:17.739468 26979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:17.740473 26979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:17.748657 26985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:17.748663 26984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:17.748816 26979 server_base.cc:1034] running on GCE node
W20260811 07:39:17.748700 26988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:17.749514 26979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:17.750545 26979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:17.751703 26979 hybrid_clock.cc:648] HybridClock initialized: now 1786433957751663 us; error 51 us; skew 500 ppm
Aug 11 07:39:17 dist-test-slave-zkpd krb5kdc[26965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433957, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:17.754477 26979 init.cc:377] Logged in from keytab as oryx/127.18.72.254@KRBTEST.COM (short username oryx)
I20260811 07:39:17.755342 26979 webserver.cc:460] Webserver started at http://127.18.72.254:39221/ using document root <none> and password file <none>
I20260811 07:39:17.755931 26979 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:17.756008 26979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:17.756232 26979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:17.758159 26979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "9e08455968ef409ba779bf717cab75d2"
format_stamp: "Formatted at 2026-08-11 07:39:17 on dist-test-slave-zkpd"
server_key: "4a14cdd7266063273851e16fa1a47247"
server_key_iv: "5fe3aaa954dcb5f676e01eb6d62069c3"
server_key_version: "encryptionkey@0"
I20260811 07:39:17.758711 26979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "9e08455968ef409ba779bf717cab75d2"
format_stamp: "Formatted at 2026-08-11 07:39:17 on dist-test-slave-zkpd"
server_key: "4a14cdd7266063273851e16fa1a47247"
server_key_iv: "5fe3aaa954dcb5f676e01eb6d62069c3"
server_key_version: "encryptionkey@0"
I20260811 07:39:17.762307 26979 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:39:17.764644 26995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:17.765805 26979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:17.765934 26979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "9e08455968ef409ba779bf717cab75d2"
format_stamp: "Formatted at 2026-08-11 07:39:17 on dist-test-slave-zkpd"
server_key: "4a14cdd7266063273851e16fa1a47247"
server_key_iv: "5fe3aaa954dcb5f676e01eb6d62069c3"
server_key_version: "encryptionkey@0"
I20260811 07:39:17.766032 26979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:17.776388 26979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:17.778944 26979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:17.779131 26979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:17.787092 26979 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:33387
I20260811 07:39:17.787109 27058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:33387 every 8 connection(s)
I20260811 07:39:17.787977 26979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:17.790635 27059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:17.794003 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 26979
I20260811 07:39:17.794150 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:17.794453 18723 external_mini_cluster.cc:1428] Setting key 603ee7fd0c4a490d127bcb458b8e586d
I20260811 07:39:17.796548 27059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: Bootstrap starting.
I20260811 07:39:17.798625 27059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:17.799343 27059 log.cc:824] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:17.801108 27059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: No bootstrap required, opened a new log
Aug 11 07:39:17 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:17.804615 27059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e08455968ef409ba779bf717cab75d2" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33387 } }
I20260811 07:39:17.804812 27059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:17.804895 27059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e08455968ef409ba779bf717cab75d2, State: Initialized, Role: FOLLOWER
I20260811 07:39:17.805294 27059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e08455968ef409ba779bf717cab75d2" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33387 } }
I20260811 07:39:17.805410 27059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:17.805492 27059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:17.805616 27059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:17.806384 27059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e08455968ef409ba779bf717cab75d2" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33387 } }
I20260811 07:39:17.806718 27059 leader_election.cc:302] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e08455968ef409ba779bf717cab75d2; no voters:
I20260811 07:39:17.807025 27059 leader_election.cc:288] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:17.807085 27065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:17.807349 27065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [term 1 LEADER]: Becoming Leader. State: Replica: 9e08455968ef409ba779bf717cab75d2, State: Running, Role: LEADER
I20260811 07:39:17.807654 27065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e08455968ef409ba779bf717cab75d2" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33387 } }
I20260811 07:39:17.808334 27059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:17.808789 27065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e08455968ef409ba779bf717cab75d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e08455968ef409ba779bf717cab75d2" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33387 } } }
I20260811 07:39:17.808909 27065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:17.809108 27065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e08455968ef409ba779bf717cab75d2. Latest consensus state: current_term: 1 leader_uuid: "9e08455968ef409ba779bf717cab75d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e08455968ef409ba779bf717cab75d2" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33387 } } }
I20260811 07:39:17.809194 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:17.795913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.18.72.254:33387)
0811 07:39:17.796306 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:17.796322 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:17.796359 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:39:17.797029 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:17.797048 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:17.797357 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:17.797673 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:17.798495 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:17.799568 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:17.800476 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:17.800930 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:17.801099 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:17.803700 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:17.803705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:17.803717 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:17.803741 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:17.805781 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:17.806212 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:17.806218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:17.806223 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:17.806350 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:17.806630 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:17.806634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:17.806636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:17.807109 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:17.807312 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:17.807951 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 07:39:17.808237 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260811 07:39:17.809216 27065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:17.809994 27072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:17.813717 27072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:17.819346 27072 catalog_manager.cc:1347] Generated new cluster ID: 917aed842f244b8798f50ff6e92baffe
I20260811 07:39:17.819442 27072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:17.832473 27072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:17.833424 27072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:17.842366 27072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: Generated new TSK 0
I20260811 07:39:17.843215 27072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab.
Entry for principal oryx/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab.
I20260811 07:39:17.881785 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33387
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab
--principal=oryx/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:18.004897 27089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:18.005131 27089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:18.005172 27089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:18.009286 27089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:18.009334 27089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:18.009371 27089 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:39:18.009403 27089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:18.014447 27089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab
--principal=oryx/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:33387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:18.015326 27089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:18.016260 27089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:18.025549 27095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:18.025576 27098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:18.025703 27089 server_base.cc:1034] running on GCE node
W20260811 07:39:18.025549 27094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:18.026213 27089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:18.026803 27089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:18.027966 27089 hybrid_clock.cc:648] HybridClock initialized: now 1786433958027945 us; error 30 us; skew 500 ppm
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433958, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:18.030696 27089 init.cc:377] Logged in from keytab as oryx/127.18.72.193@KRBTEST.COM (short username oryx)
I20260811 07:39:18.031567 27089 webserver.cc:460] Webserver started at http://127.18.72.193:34425/ using document root <none> and password file <none>
I20260811 07:39:18.032176 27089 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:18.032238 27089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:18.032469 27089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:18.034401 27089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "0f5848513fbd48129664f5be11e46838"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "1ce2b362c3076ca532d7fdcb816b95f2"
server_key_iv: "e53d1b10152544e7e26e9bb06baeadda"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.034960 27089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "0f5848513fbd48129664f5be11e46838"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "1ce2b362c3076ca532d7fdcb816b95f2"
server_key_iv: "e53d1b10152544e7e26e9bb06baeadda"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.038553 27089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 07:39:18.040894 27105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:18.042019 27089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:18.042147 27089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "0f5848513fbd48129664f5be11e46838"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "1ce2b362c3076ca532d7fdcb816b95f2"
server_key_iv: "e53d1b10152544e7e26e9bb06baeadda"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.042248 27089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:18.061985 27089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:18.064751 27089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:18.064949 27089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:18.066344 27089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:18.066417 27089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:18.066490 27089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:18.066533 27089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433958, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:18.078118 27089 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:38495
I20260811 07:39:18.078217 27232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:38495 every 8 connection(s)
I20260811 07:39:18.078966 27089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:18.079669 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 27089
I20260811 07:39:18.079816 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:18.080168 18723 external_mini_cluster.cc:1428] Setting key 36c89948e92d468f18fdd7e1ab41bfd8
I20260811 07:39:18.080937 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.067326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:37179 (local address 127.18.72.254:33387)
0811 07:39:18.067457 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.067461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.068454 (+ 993us) server_negotiation.cc:407] Connection header received
0811 07:39:18.069491 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.069495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.069565 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.069665 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:18.071005 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.071984 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.073516 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.073810 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.073908 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.076890 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:18.076893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:18.076896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:18.076922 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:18.078667 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.079223 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.079227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.079229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.079321 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.079672 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.079677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.079678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.079941 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:18.080037 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.080627 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.080773 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:39:18.081561 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.067818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33387 (local address 127.18.72.193:37179)
0811 07:39:18.068299 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.068350 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.069205 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.069942 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.069952 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.070305 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:18.070844 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.070859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.072141 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.072144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.073377 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.073389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.074088 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.074093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.074244 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.074791 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:18.074810 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:18.076716 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:18.078787 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.078792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.078806 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.079113 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.079432 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.079435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.079437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.079574 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.080096 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:18.080101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:18.080366 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.080517 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.080736 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":125,"threads_started":1}
I20260811 07:39:18.084613 27233 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33387
I20260811 07:39:18.084820 27233 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:18.085279 27233 heartbeater.cc:511] Master 127.18.72.254:33387 requested a full tablet report, sending...
I20260811 07:39:18.086941 27011 ts_manager.cc:194] Registered new tserver with Master: 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495)
I20260811 07:39:18.088673 27011 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:37179
WARNING: no policy specified for oryx/127.18.72.194@KRBTEST.COM; defaulting to no policy
I20260811 07:39:18.093778 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.087037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:40021 (local address 127.18.72.254:33387)
0811 07:39:18.087157 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.087162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.087174 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:18.087203 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.087206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.087246 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.087326 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:18.087739 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.088493 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.089431 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.089627 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.089726 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.090713 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:18.090717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:18.090719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:18.090742 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:18.092190 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.092607 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.092611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.092613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.092689 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.092983 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.092987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.092989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.093151 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:18.093255 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.093508 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.093617 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":179,"server-negotiator.queue_time_us":46}
I20260811 07:39:18.093786 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.086743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33387 (local address 127.18.72.193:40021)
0811 07:39:18.086997 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.087011 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.087104 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.087342 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.087345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.087540 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:18.087628 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.087633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.088614 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.088617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.089227 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.089236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.089732 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.089736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.089829 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.090274 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:18.090284 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:18.090565 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:18.092295 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.092299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.092301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.092497 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.092779 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.092783 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.092785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.092874 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.093247 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:18.093250 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:18.093340 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.093454 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.093584 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":108,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab.
Entry for principal oryx/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab.
I20260811 07:39:18.112381 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33387
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab
--principal=oryx/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:18.226449 27243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:18.226713 27243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:18.226759 27243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:18.230690 27243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:18.230751 27243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:18.230788 27243 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:39:18.230820 27243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:18.235932 27243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab
--principal=oryx/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:33387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:18.236884 27243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:18.237833 27243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:18.247354 27252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:18.247400 27250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:18.247349 27249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:18.247718 27243 server_base.cc:1034] running on GCE node
I20260811 07:39:18.248164 27243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:18.248824 27243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:18.250021 27243 hybrid_clock.cc:648] HybridClock initialized: now 1786433958250005 us; error 26 us; skew 500 ppm
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433958, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:18.252766 27243 init.cc:377] Logged in from keytab as oryx/127.18.72.194@KRBTEST.COM (short username oryx)
I20260811 07:39:18.253538 27243 webserver.cc:460] Webserver started at http://127.18.72.194:38661/ using document root <none> and password file <none>
I20260811 07:39:18.254066 27243 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:18.254124 27243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:18.254281 27243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:18.256306 27243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "959392cac7cd490eadc2f337bd4d5e19"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "cbab653b235e62128b3e693b997aca7b"
server_key_iv: "27ef9ffe36e2f097654fc335d340119a"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.256812 27243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "959392cac7cd490eadc2f337bd4d5e19"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "cbab653b235e62128b3e693b997aca7b"
server_key_iv: "27ef9ffe36e2f097654fc335d340119a"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.260404 27243 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:39:18.262661 27259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:18.263800 27243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:18.263967 27243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "959392cac7cd490eadc2f337bd4d5e19"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "cbab653b235e62128b3e693b997aca7b"
server_key_iv: "27ef9ffe36e2f097654fc335d340119a"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.264057 27243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:18.277467 27243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:18.280445 27243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:18.280647 27243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:18.282128 27243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:18.282202 27243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:18.282272 27243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:18.282323 27243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433958, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:18.293813 27243 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:41003
I20260811 07:39:18.293922 27379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:41003 every 8 connection(s)
I20260811 07:39:18.294968 27243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:18.296682 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.282987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:46781 (local address 127.18.72.254:33387)
0811 07:39:18.283150 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.283156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.284108 (+ 952us) server_negotiation.cc:407] Connection header received
0811 07:39:18.285163 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.285168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.285227 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.285284 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:18.286826 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.287737 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.289088 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.289339 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.289466 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.292675 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:18.292679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:18.292682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:18.292718 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:18.294433 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.295000 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.295007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.295009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.295106 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.295519 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.295522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.295524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.295749 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:18.295880 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.296459 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.296566 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 07:39:18.297479 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.283407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33387 (local address 127.18.72.194:46781)
0811 07:39:18.283943 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.283997 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.284894 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.285713 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.285723 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.286173 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:18.286656 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.286669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.287927 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.287934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.288950 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.288961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.289545 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.289552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.289819 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.290525 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:18.290546 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:18.292496 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:18.294569 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.294574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.294586 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.294870 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.295209 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.295212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.295213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.295390 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.295973 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:18.295979 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:18.296214 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.296370 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.296622 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":126,"threads_started":1}
I20260811 07:39:18.298485 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 27243
I20260811 07:39:18.298635 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:18.298875 27380 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33387
I20260811 07:39:18.299050 18723 external_mini_cluster.cc:1428] Setting key e1814f1109744838a1144311b350e051
I20260811 07:39:18.299125 27380 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:18.300338 27380 heartbeater.cc:511] Master 127.18.72.254:33387 requested a full tablet report, sending...
I20260811 07:39:18.301533 27011 ts_manager.cc:194] Registered new tserver with Master: 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003)
I20260811 07:39:18.302240 27011 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:46781
I20260811 07:39:18.310053 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.302009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33387 (local address 127.18.72.194:58129)
0811 07:39:18.302312 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.302326 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.302427 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.302720 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.302723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.302944 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:18.303062 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.303069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.304118 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.304121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.304790 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.304800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.305421 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.305424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.305490 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.305997 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:18.306017 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:18.306314 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:18.308338 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.308340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.308342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.308523 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.308838 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.308840 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.308841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.308938 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.309459 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:18.309462 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:18.309536 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.309699 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.309870 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":128,"threads_started":1}
I20260811 07:39:18.310138 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.302027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:58129 (local address 127.18.72.254:33387)
0811 07:39:18.302131 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.302135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.302414 (+ 279us) server_negotiation.cc:407] Connection header received
0811 07:39:18.302533 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.302539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.302597 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.302685 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:18.303201 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.303959 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.304911 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.305305 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.305432 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.306455 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:18.306461 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:18.306465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:18.306495 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:18.308227 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.308625 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.308629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.308631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.308735 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.309048 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.309051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.309053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.309347 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:18.309460 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.309881 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.309989 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for oryx/127.18.72.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab.
Entry for principal oryx/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab.
I20260811 07:39:18.330559 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33387
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab
--principal=oryx/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:18.446851 27391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:18.447105 27391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:18.447150 27391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:18.451072 27391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:18.451125 27391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:18.451162 27391 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 07:39:18.451195 27391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:18.456285 27391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab
--principal=oryx/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.72.254:33387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:18.457424 27391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:18.458523 27391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:18.468210 27401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:18.468518 27391 server_base.cc:1034] running on GCE node
W20260811 07:39:18.468261 27398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:18.468263 27396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:18.469062 27391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:18.469664 27391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:18.470873 27391 hybrid_clock.cc:648] HybridClock initialized: now 1786433958470839 us; error 47 us; skew 500 ppm
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433958, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:18.473730 27391 init.cc:377] Logged in from keytab as oryx/127.18.72.195@KRBTEST.COM (short username oryx)
I20260811 07:39:18.474623 27391 webserver.cc:460] Webserver started at http://127.18.72.195:44403/ using document root <none> and password file <none>
I20260811 07:39:18.475200 27391 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:18.475296 27391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:18.475538 27391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:18.477702 27391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "be064db7576d4167a45e34ebcfe725d3"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "51e25cd21d5dfc4a55cbb335cc5ef9e6"
server_key_iv: "7449226d8cdffcbe0579ef21d4c79b40"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.478291 27391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "be064db7576d4167a45e34ebcfe725d3"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "51e25cd21d5dfc4a55cbb335cc5ef9e6"
server_key_iv: "7449226d8cdffcbe0579ef21d4c79b40"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.482045 27391 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:39:18.484632 27408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:18.485898 27391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:18.486034 27391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "be064db7576d4167a45e34ebcfe725d3"
format_stamp: "Formatted at 2026-08-11 07:39:18 on dist-test-slave-zkpd"
server_key: "51e25cd21d5dfc4a55cbb335cc5ef9e6"
server_key_iv: "7449226d8cdffcbe0579ef21d4c79b40"
server_key_version: "encryptionkey@0"
I20260811 07:39:18.486142 27391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:18.508986 27391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:18.511965 27391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:18.512178 27391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:18.513639 27391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:18.513739 27391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:18.513813 27391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:18.513866 27391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433958, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:18.525403 27391 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:42529
I20260811 07:39:18.526242 27391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:18.527078 27538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:42529 every 8 connection(s)
I20260811 07:39:18.527993 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.514506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:42513 (local address 127.18.72.254:33387)
0811 07:39:18.514634 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.514638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.515701 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 07:39:18.516676 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.516679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.516739 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.516827 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:18.518170 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.518929 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.520229 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.520458 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.520572 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.524296 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:18.524299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:18.524302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:18.524336 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:18.525851 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.526371 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.526374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.526375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.526450 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.526848 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.526852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.526853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.527083 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:18.527196 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.527664 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.527786 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:39:18.528139 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 27391
I20260811 07:39:18.528280 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:18.528563 18723 external_mini_cluster.cc:1428] Setting key 7bc876f83777d6607fe1991fe674d3cc
I20260811 07:39:18.528741 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.515044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33387 (local address 127.18.72.195:42513)
0811 07:39:18.515548 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.515591 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.516470 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.517151 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.517162 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.517506 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:18.518027 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.518041 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.519064 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.519067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.520102 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.520112 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.520613 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.520617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.520780 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.521418 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:18.521442 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:18.524126 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:18.525969 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.525975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.525989 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.526246 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.526543 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.526547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.526549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.526729 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.527193 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:18.527200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:18.527426 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.527600 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.527969 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":154,"threads_started":1}
I20260811 07:39:18.531793 27539 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33387
I20260811 07:39:18.531960 27539 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:18.532455 27539 heartbeater.cc:511] Master 127.18.72.254:33387 requested a full tablet report, sending...
I20260811 07:39:18.533516 27010 ts_manager.cc:194] Registered new tserver with Master: be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
I20260811 07:39:18.534242 27010 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:42513
I20260811 07:39:18.539152 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.532084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:58319 (local address 127.18.72.254:33387)
0811 07:39:18.532229 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.532233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.532470 (+ 237us) server_negotiation.cc:407] Connection header received
0811 07:39:18.532693 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.532696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.532739 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.532816 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:18.533282 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.533885 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.534850 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.535080 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.535156 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.536409 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:18.536412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:18.536414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:18.536434 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:18.537858 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.538260 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.538262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.538263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.538336 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.538566 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.538568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.538569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.538716 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:18.538805 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.538972 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.539054 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:39:18.539175 27546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.532105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33387 (local address 127.18.72.195:58319)
0811 07:39:18.532374 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.532389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.532490 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.532855 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.532859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.533069 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:18.533181 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.533187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.534012 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.534015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.534722 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.534754 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.535192 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.535196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.535326 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.535840 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:18.535853 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:18.536281 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:18.537961 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.537963 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.537965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.538148 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.538415 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:18.538417 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:18.538419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:18.538492 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:18.538793 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:18.538795 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:18.538864 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.538941 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.539038 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260811 07:39:18.542275 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433958, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 07:39:18.567664 27062 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:18.561451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59262 (local address 127.18.72.254:33387)
0811 07:39:18.561596 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.561599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.561613 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:18.561655 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.561658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.561696 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.561789 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:18.562508 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.563037 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.563926 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.564153 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.564262 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.567546 (+ 3284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59262: BlockingRecv error: recv got EOF from 127.0.0.1:59262 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 07:39:18 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433958, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:18.576865 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.569113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.18.72.254:33387)
0811 07:39:18.569227 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.569231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.569320 (+ 89us) server_negotiation.cc:407] Connection header received
0811 07:39:18.569448 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.569450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.569489 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.569556 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:18.570104 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.570682 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.571427 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.571628 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.571703 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.573953 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:18.573956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:18.573958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:18.573981 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:18.575379 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.575792 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.575795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.575796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.575867 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:18.576169 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:18.576172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:18.576173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:18.576350 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:18.576451 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.576644 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.576738 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:39:18.579710 27010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:39:18.582823 27010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:18.598086 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.591984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35110 (local address 127.18.72.193:38495)
0811 07:39:18.592281 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.592288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.592619 (+ 331us) server_negotiation.cc:407] Connection header received
0811 07:39:18.592899 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.592905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.593071 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.593212 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:18.593598 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.594803 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.596532 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.597263 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.597329 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.597470 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.597824 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.597933 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":99,"threads_started":1}
I20260811 07:39:18.598104 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.592169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42529 (local address 127.0.0.1:45240)
0811 07:39:18.593481 (+ 1312us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.593497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.593575 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.594243 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.594246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.594264 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:18.594363 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.594369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.595602 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.595605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.596460 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.596473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.597477 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.597481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.597604 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.597645 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.597761 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.597933 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"mutex_wait_us":5,"thread_start_us":84,"threads_started":1}
I20260811 07:39:18.598101 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.591551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:38495 (local address 127.0.0.1:35110)
0811 07:39:18.592475 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.592528 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.592676 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.593221 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.593228 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.593253 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:18.593433 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.593443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.594918 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.594941 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.596388 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.596400 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.597561 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.597564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.597645 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.597662 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.597761 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.597933 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":103,"spinlock_wait_cycles":2432,"thread_start_us":109,"threads_started":1}
I20260811 07:39:18.598587 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.592467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45240 (local address 127.18.72.195:42529)
0811 07:39:18.593805 (+ 1338us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.593814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.593832 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:39:18.593891 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.593894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.594097 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.594266 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:18.594480 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.595453 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.596613 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.597346 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.597458 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.597610 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.598338 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.598449 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1251,"thread_start_us":139,"threads_started":1}
I20260811 07:39:18.598981 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.591462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41003 (local address 127.0.0.1:35386)
0811 07:39:18.592475 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.592528 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.592664 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.593249 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.593252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.593270 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:18.593438 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.593444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.595645 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.595653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.596836 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.596846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.598090 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.598095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.598713 (+ 618us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.598740 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.598833 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.598888 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":58,"thread_start_us":56,"threads_started":1}
I20260811 07:39:18.599272 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.591605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.18.72.194:41003)
0811 07:39:18.592205 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.592221 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.592807 (+ 586us) server_negotiation.cc:407] Connection header received
0811 07:39:18.592920 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.592927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.593090 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.593257 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:18.593562 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.594796 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.596961 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.597769 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.597880 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.598024 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.598862 (+ 838us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.598957 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":108,"threads_started":1}
I20260811 07:39:18.601389 27158 tablet_service.cc:1467] Processing CreateTablet for tablet 5dee322b60bc4f219c2d7c8f321bcbef (DEFAULT_TABLE table=test-table [id=eb6d4087207146e19ce152dcf8be62f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:18.601722 27310 tablet_service.cc:1467] Processing CreateTablet for tablet 5dee322b60bc4f219c2d7c8f321bcbef (DEFAULT_TABLE table=test-table [id=eb6d4087207146e19ce152dcf8be62f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:18.602419 27158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dee322b60bc4f219c2d7c8f321bcbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:18.602679 27310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dee322b60bc4f219c2d7c8f321bcbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:18.605847 27465 tablet_service.cc:1467] Processing CreateTablet for tablet 5dee322b60bc4f219c2d7c8f321bcbef (DEFAULT_TABLE table=test-table [id=eb6d4087207146e19ce152dcf8be62f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:18.606840 27465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dee322b60bc4f219c2d7c8f321bcbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:18.609319 27569 tablet_bootstrap.cc:492] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Bootstrap starting.
I20260811 07:39:18.610735 27570 tablet_bootstrap.cc:492] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Bootstrap starting.
I20260811 07:39:18.611753 27569 tablet_bootstrap.cc:654] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:18.612666 27569 log.cc:824] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:18.612797 27570 tablet_bootstrap.cc:654] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:18.613731 27570 log.cc:824] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:18.614764 27572 tablet_bootstrap.cc:492] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Bootstrap starting.
I20260811 07:39:18.614814 27569 tablet_bootstrap.cc:492] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: No bootstrap required, opened a new log
I20260811 07:39:18.614991 27569 ts_tablet_manager.cc:1397] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 07:39:18.615836 27570 tablet_bootstrap.cc:492] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: No bootstrap required, opened a new log
I20260811 07:39:18.616053 27570 ts_tablet_manager.cc:1397] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 07:39:18.616529 27572 tablet_bootstrap.cc:654] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:18.617173 27572 log.cc:824] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:18.618759 27572 tablet_bootstrap.cc:492] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: No bootstrap required, opened a new log
I20260811 07:39:18.618958 27572 ts_tablet_manager.cc:1397] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 07:39:18.619120 27569 raft_consensus.cc:348] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.619395 27569 raft_consensus.cc:374] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:18.619464 27569 raft_consensus.cc:729] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959392cac7cd490eadc2f337bd4d5e19, State: Initialized, Role: FOLLOWER
I20260811 07:39:18.619964 27569 consensus_queue.cc:260] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.620616 27570 raft_consensus.cc:348] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.620862 27570 raft_consensus.cc:374] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:18.620924 27570 raft_consensus.cc:729] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5848513fbd48129664f5be11e46838, State: Initialized, Role: FOLLOWER
I20260811 07:39:18.621006 27569 ts_tablet_manager.cc:1428] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:39:18.621387 27380 heartbeater.cc:503] Master 127.18.72.254:33387 was elected leader, sending a full tablet report...
I20260811 07:39:18.621367 27570 consensus_queue.cc:260] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.621943 27233 heartbeater.cc:503] Master 127.18.72.254:33387 was elected leader, sending a full tablet report...
I20260811 07:39:18.622254 27570 ts_tablet_manager.cc:1428] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:39:18.622767 27572 raft_consensus.cc:348] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.623011 27572 raft_consensus.cc:374] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:18.623070 27572 raft_consensus.cc:729] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be064db7576d4167a45e34ebcfe725d3, State: Initialized, Role: FOLLOWER
I20260811 07:39:18.623453 27572 consensus_queue.cc:260] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.624156 27539 heartbeater.cc:503] Master 127.18.72.254:33387 was elected leader, sending a full tablet report...
I20260811 07:39:18.624460 27572 ts_tablet_manager.cc:1428] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 07:39:18.777272 27540 tablet.cc:2369] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:18.787655 27576 raft_consensus.cc:482] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:18.787923 27576 raft_consensus.cc:504] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.789116 27576 leader_election.cc:288] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495), be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
I20260811 07:39:18.793889 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.789800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:44533 (local address 127.18.72.193:38495)
0811 07:39:18.789938 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.789943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.789957 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:18.790010 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.790013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.790078 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.790145 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:18.790513 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.791265 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.792738 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.793356 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.793470 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.793519 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.793680 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.793770 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:39:18.793893 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.789319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:38495 (local address 127.18.72.194:44533)
0811 07:39:18.789592 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.789607 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.789694 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.790250 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.790253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.790278 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:18.790397 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.790403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.791373 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.791376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.792608 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.792619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.793465 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.793468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.793549 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.793569 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.793658 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.793758 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":57,"spinlock_wait_cycles":1408}
I20260811 07:39:18.794844 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.789373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42529 (local address 127.18.72.194:34383)
0811 07:39:18.789759 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:18.789774 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:18.789893 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:18.790185 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:18.790188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:18.790204 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:18.790359 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.790364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.791224 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.791229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.792608 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:18.792619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.794433 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.794436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:18.794538 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.794559 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 07:39:18.794632 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:39:18.794695 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":50,"thread_start_us":104,"threads_started":1}
I20260811 07:39:18.794777 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dee322b60bc4f219c2d7c8f321bcbef" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5848513fbd48129664f5be11e46838" is_pre_election: true
I20260811 07:39:18.794875 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.789493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:34383 (local address 127.18.72.195:42529)
0811 07:39:18.789639 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.789644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.789901 (+ 257us) server_negotiation.cc:407] Connection header received
0811 07:39:18.789990 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.789993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.790043 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.790130 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:18.790474 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.791118 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.793574 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.794216 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.794322 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.794368 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.794673 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.794750 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:39:18.795132 27183 raft_consensus.cc:2454] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 0.
I20260811 07:39:18.795535 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dee322b60bc4f219c2d7c8f321bcbef" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be064db7576d4167a45e34ebcfe725d3" is_pre_election: true
I20260811 07:39:18.795663 27263 leader_election.cc:302] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, 959392cac7cd490eadc2f337bd4d5e19; no voters:
I20260811 07:39:18.795823 27488 raft_consensus.cc:2454] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 0.
I20260811 07:39:18.795975 27576 raft_consensus.cc:2781] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:18.796058 27576 raft_consensus.cc:482] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:18.796104 27576 raft_consensus.cc:3037] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:18.797538 27576 raft_consensus.cc:504] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.797852 27576 leader_election.cc:288] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 election: Requested vote from peers 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495), be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
W20260811 07:39:18.798069 27381 tablet.cc:2369] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:18.798166 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dee322b60bc4f219c2d7c8f321bcbef" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be064db7576d4167a45e34ebcfe725d3"
I20260811 07:39:18.798156 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dee322b60bc4f219c2d7c8f321bcbef" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5848513fbd48129664f5be11e46838"
I20260811 07:39:18.798267 27488 raft_consensus.cc:3037] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:18.798280 27183 raft_consensus.cc:3037] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:18.799564 27183 raft_consensus.cc:2454] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 1.
I20260811 07:39:18.799944 27488 raft_consensus.cc:2454] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 1.
I20260811 07:39:18.799965 27263 leader_election.cc:302] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, 959392cac7cd490eadc2f337bd4d5e19; no voters:
I20260811 07:39:18.800124 27576 raft_consensus.cc:2781] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:18.800408 27576 raft_consensus.cc:686] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 1 LEADER]: Becoming Leader. State: Replica: 959392cac7cd490eadc2f337bd4d5e19, State: Running, Role: LEADER
I20260811 07:39:18.800812 27576 consensus_queue.cc:237] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.803704 27010 catalog_manager.cc:5495] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194). New cstate: current_term: 1 leader_uuid: "959392cac7cd490eadc2f337bd4d5e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20260811 07:39:18.830701 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:18.827023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.18.72.194:41003)
0811 07:39:18.827142 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:18.827147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:18.827161 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:18.827289 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:18.827291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:18.827345 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:18.827454 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:18.827711 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.828293 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.829150 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:18.829407 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:18.829550 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:18.829852 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:18.830274 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:18.830400 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:18.830464 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:39:18.830519 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 07:39:18.830891 27234 tablet.cc:2369] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:18.836436 27488 raft_consensus.cc:1261] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Refusing update from remote peer 959392cac7cd490eadc2f337bd4d5e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:18.836439 27183 raft_consensus.cc:1261] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Refusing update from remote peer 959392cac7cd490eadc2f337bd4d5e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:18.837149 27576 consensus_queue.cc:1035] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:18.837332 27581 consensus_queue.cc:1035] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:18.845206 27586 mvcc.cc:205] Tried to move back new op lower bound from 7317233495387095040 to 7317233495250681856. Current Snapshot: MvccSnapshot[applied={T|T < 7317233495387095040}]
I20260811 07:39:18.845455 27585 mvcc.cc:205] Tried to move back new op lower bound from 7317233495387095040 to 7317233495250681856. Current Snapshot: MvccSnapshot[applied={T|T < 7317233495387095040}]
I20260811 07:39:18.855647 27010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:18.855823 27010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:18.858232 27010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: Sending DeleteTablet for 3 replicas of tablet 5dee322b60bc4f219c2d7c8f321bcbef
I20260811 07:39:18.858959 27158 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dee322b60bc4f219c2d7c8f321bcbef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:18 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35110
I20260811 07:39:18.859012 27465 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dee322b60bc4f219c2d7c8f321bcbef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:18 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45240
I20260811 07:39:18.859079 27310 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dee322b60bc4f219c2d7c8f321bcbef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:18 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35386
I20260811 07:39:18.859462 27595 tablet_replica.cc:330] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: stopping tablet replica
I20260811 07:39:18.859632 27594 tablet_replica.cc:330] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: stopping tablet replica
I20260811 07:39:18.859769 27595 raft_consensus.cc:2229] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:18.860028 27594 raft_consensus.cc:2229] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:18.860114 27595 raft_consensus.cc:2258] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:18.860173 27010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 07:39:18.860564 27594 raft_consensus.cc:2258] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 07:39:18.860603 27010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:18.861606 27595 ts_tablet_manager.cc:1905] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:18.861661 27596 tablet_replica.cc:330] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: stopping tablet replica
I20260811 07:39:18.861937 27596 raft_consensus.cc:2229] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:39:18.862308 27596 raft_consensus.cc:2258] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:18.863148 27594 ts_tablet_manager.cc:1905] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:18.865394 27595 ts_tablet_manager.cc:1918] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:18.865495 27595 log.cc:1196] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/5dee322b60bc4f219c2d7c8f321bcbef
I20260811 07:39:18.865830 27595 ts_tablet_manager.cc:1939] T 5dee322b60bc4f219c2d7c8f321bcbef P be064db7576d4167a45e34ebcfe725d3: Deleting consensus metadata
I20260811 07:39:18.866238 27594 ts_tablet_manager.cc:1918] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:18.866410 27594 log.cc:1196] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/5dee322b60bc4f219c2d7c8f321bcbef
I20260811 07:39:18.866807 27594 ts_tablet_manager.cc:1939] T 5dee322b60bc4f219c2d7c8f321bcbef P 0f5848513fbd48129664f5be11e46838: Deleting consensus metadata
I20260811 07:39:18.866963 27158 tablet_service.cc:1467] Processing CreateTablet for tablet e467f189efab4d6099c76a28ba54628c (DEFAULT_TABLE table=test-table [id=577ee887684c4ae09d83247a95fc5b5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:18.867353 27158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e467f189efab4d6099c76a28ba54628c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:18.867370 26997 catalog_manager.cc:4841] TS be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529): tablet 5dee322b60bc4f219c2d7c8f321bcbef (table test-table [id=eb6d4087207146e19ce152dcf8be62f2]) successfully deleted
I20260811 07:39:18.868312 27465 tablet_service.cc:1467] Processing CreateTablet for tablet e467f189efab4d6099c76a28ba54628c (DEFAULT_TABLE table=test-table [id=577ee887684c4ae09d83247a95fc5b5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:18.868562 27465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e467f189efab4d6099c76a28ba54628c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:18.869490 27570 tablet_bootstrap.cc:492] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: Bootstrap starting.
I20260811 07:39:18.870457 27570 tablet_bootstrap.cc:654] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:18.870713 27572 tablet_bootstrap.cc:492] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: Bootstrap starting.
I20260811 07:39:18.870945 26999 catalog_manager.cc:4841] TS 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495): tablet 5dee322b60bc4f219c2d7c8f321bcbef (table test-table [id=eb6d4087207146e19ce152dcf8be62f2]) successfully deleted
I20260811 07:39:18.871387 27310 tablet_service.cc:1467] Processing CreateTablet for tablet e467f189efab4d6099c76a28ba54628c (DEFAULT_TABLE table=test-table [id=577ee887684c4ae09d83247a95fc5b5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:18.871574 27572 tablet_bootstrap.cc:654] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:18.871683 27310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e467f189efab4d6099c76a28ba54628c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:18.872294 27570 tablet_bootstrap.cc:492] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: No bootstrap required, opened a new log
I20260811 07:39:18.872390 27570 ts_tablet_manager.cc:1397] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:39:18.872709 27572 tablet_bootstrap.cc:492] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: No bootstrap required, opened a new log
I20260811 07:39:18.872788 27572 ts_tablet_manager.cc:1397] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 07:39:18.872859 27570 raft_consensus.cc:348] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.872960 27570 raft_consensus.cc:374] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:18.872988 27570 raft_consensus.cc:729] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5848513fbd48129664f5be11e46838, State: Initialized, Role: FOLLOWER
I20260811 07:39:18.873098 27570 consensus_queue.cc:260] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.873200 27572 raft_consensus.cc:348] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.873296 27572 raft_consensus.cc:374] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:18.873325 27572 raft_consensus.cc:729] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be064db7576d4167a45e34ebcfe725d3, State: Initialized, Role: FOLLOWER
I20260811 07:39:18.873318 27570 ts_tablet_manager.cc:1428] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 07:39:18.873435 27572 consensus_queue.cc:260] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.873658 27572 ts_tablet_manager.cc:1428] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 07:39:18.874828 27569 tablet_bootstrap.cc:492] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: Bootstrap starting.
I20260811 07:39:18.875825 27569 tablet_bootstrap.cc:654] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:18.877126 27569 tablet_bootstrap.cc:492] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: No bootstrap required, opened a new log
I20260811 07:39:18.877215 27569 ts_tablet_manager.cc:1397] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:18.877645 27569 raft_consensus.cc:348] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.877745 27569 raft_consensus.cc:374] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:18.877777 27569 raft_consensus.cc:729] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959392cac7cd490eadc2f337bd4d5e19, State: Initialized, Role: FOLLOWER
I20260811 07:39:18.877899 27569 consensus_queue.cc:260] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:18.878051 27596 ts_tablet_manager.cc:1905] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:18.878135 27569 ts_tablet_manager.cc:1428] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:18.880518 27596 ts_tablet_manager.cc:1918] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:18.880626 27596 log.cc:1196] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/5dee322b60bc4f219c2d7c8f321bcbef
I20260811 07:39:18.880975 27596 ts_tablet_manager.cc:1939] T 5dee322b60bc4f219c2d7c8f321bcbef P 959392cac7cd490eadc2f337bd4d5e19: Deleting consensus metadata
I20260811 07:39:18.882036 26997 catalog_manager.cc:4841] TS 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003): tablet 5dee322b60bc4f219c2d7c8f321bcbef (table test-table [id=eb6d4087207146e19ce152dcf8be62f2]) successfully deleted
I20260811 07:39:19.252288 27577 raft_consensus.cc:482] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:19.252481 27577 raft_consensus.cc:504] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.253935 27577 leader_election.cc:288] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003), be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
I20260811 07:39:19.258915 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.254758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:40229 (local address 127.18.72.195:42529)
0811 07:39:19.254887 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.254893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.254947 (+ 54us) server_negotiation.cc:407] Connection header received
0811 07:39:19.255131 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.255134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.255189 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.255267 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:19.255504 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.256254 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.257586 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.258186 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.258310 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.258366 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.258694 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.258791 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:39:19.258915 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.254314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:60945 (local address 127.18.72.194:41003)
0811 07:39:19.254474 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.254478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.254565 (+ 87us) server_negotiation.cc:407] Connection header received
0811 07:39:19.254763 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.254767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.254831 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.254886 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:19.255218 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.256051 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.257586 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.258185 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.258310 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.258367 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.258694 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.258779 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:39:19.258984 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.254168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41003 (local address 127.18.72.193:60945)
0811 07:39:19.254467 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:19.254485 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:19.254602 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:19.255024 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:19.255026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:19.255043 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:19.255122 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.255127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.256153 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.256156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.257444 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.257456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.258311 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.258314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.258510 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.258574 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 07:39:19.258655 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 07:39:19.258779 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":96,"thread_start_us":119,"threads_started":1}
I20260811 07:39:19.258992 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.254587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42529 (local address 127.18.72.193:40229)
0811 07:39:19.254862 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:19.254876 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:19.255007 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:19.255293 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:19.255295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:19.255309 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:19.255396 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.255400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.256371 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.256375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.257465 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.257472 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.258311 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.258315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.258532 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.258580 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 07:39:19.258655 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 07:39:19.258779 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":111,"thread_start_us":104,"threads_started":1}
I20260811 07:39:19.259541 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e467f189efab4d6099c76a28ba54628c" candidate_uuid: "0f5848513fbd48129664f5be11e46838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be064db7576d4167a45e34ebcfe725d3" is_pre_election: true
I20260811 07:39:19.259691 27488 raft_consensus.cc:2454] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f5848513fbd48129664f5be11e46838 in term 0.
I20260811 07:39:19.260097 27107 leader_election.cc:302] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, be064db7576d4167a45e34ebcfe725d3; no voters:
I20260811 07:39:19.260257 27330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e467f189efab4d6099c76a28ba54628c" candidate_uuid: "0f5848513fbd48129664f5be11e46838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959392cac7cd490eadc2f337bd4d5e19" is_pre_election: true
I20260811 07:39:19.260375 27577 raft_consensus.cc:2781] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:19.260458 27577 raft_consensus.cc:482] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:19.260507 27577 raft_consensus.cc:3037] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:19.260527 27330 raft_consensus.cc:2454] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f5848513fbd48129664f5be11e46838 in term 0.
I20260811 07:39:19.261601 27577 raft_consensus.cc:504] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.261940 27577 leader_election.cc:288] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [CANDIDATE]: Term 1 election: Requested vote from peers 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003), be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
I20260811 07:39:19.262277 27330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e467f189efab4d6099c76a28ba54628c" candidate_uuid: "0f5848513fbd48129664f5be11e46838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959392cac7cd490eadc2f337bd4d5e19"
I20260811 07:39:19.262425 27330 raft_consensus.cc:3037] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:19.262382 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e467f189efab4d6099c76a28ba54628c" candidate_uuid: "0f5848513fbd48129664f5be11e46838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be064db7576d4167a45e34ebcfe725d3"
I20260811 07:39:19.262517 27488 raft_consensus.cc:3037] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:19.263435 27330 raft_consensus.cc:2454] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f5848513fbd48129664f5be11e46838 in term 1.
I20260811 07:39:19.263440 27488 raft_consensus.cc:2454] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f5848513fbd48129664f5be11e46838 in term 1.
I20260811 07:39:19.263720 27107 leader_election.cc:302] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, 959392cac7cd490eadc2f337bd4d5e19; no voters:
I20260811 07:39:19.263958 27577 raft_consensus.cc:2781] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:19.264231 27577 raft_consensus.cc:686] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 1 LEADER]: Becoming Leader. State: Replica: 0f5848513fbd48129664f5be11e46838, State: Running, Role: LEADER
I20260811 07:39:19.264500 27577 consensus_queue.cc:237] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.266849 27010 catalog_manager.cc:5495] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f5848513fbd48129664f5be11e46838 (127.18.72.193). New cstate: current_term: 1 leader_uuid: "0f5848513fbd48129664f5be11e46838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
Aug 11 07:39:19 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433957, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.254@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:19.365840 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.357766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33387 (local address 127.0.0.1:59280)
0811 07:39:19.358070 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:19.358082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:19.358190 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:19.358437 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:19.358439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:19.358837 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:19.358934 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.358940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.359657 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.359659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.360288 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.360297 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.360717 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.360720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.360781 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.361296 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:19.361307 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:19.362664 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:19.364466 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:19.364474 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:19.364480 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:19.364699 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:19.364966 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:19.364968 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:19.364969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:19.365045 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:19.365387 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:19.365392 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:19.365491 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 07:39:19.365581 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 07:39:19.365701 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":124,"threads_started":1}
I20260811 07:39:19.365922 27610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.357821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59280 (local address 127.18.72.254:33387)
0811 07:39:19.358085 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.358104 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.358163 (+ 59us) server_negotiation.cc:407] Connection header received
0811 07:39:19.358267 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.358270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.358319 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.358400 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:19.359036 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.359567 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.360396 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.360618 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.360696 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.362803 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:19.362806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:19.362808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:19.362829 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:19.364377 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:19.364801 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:19.364804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:19.364805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:19.364883 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:19.365122 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:19.365124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:19.365125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:19.365292 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:19.365367 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.365637 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.365762 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260811 07:39:19.370110 27010 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:59280:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 07:39:19.370879 27010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:19.375803 27158 tablet_service.cc:1467] Processing CreateTablet for tablet ebe5ea1817f14a7cac64c356048bdd62 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a5c4714727643c0b03ac3c2cbbceeab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:39:19.376027 27465 tablet_service.cc:1467] Processing CreateTablet for tablet ebe5ea1817f14a7cac64c356048bdd62 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a5c4714727643c0b03ac3c2cbbceeab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:39:19.376027 27310 tablet_service.cc:1467] Processing CreateTablet for tablet ebe5ea1817f14a7cac64c356048bdd62 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a5c4714727643c0b03ac3c2cbbceeab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 07:39:19.376246 27158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe5ea1817f14a7cac64c356048bdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:19.376410 27465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe5ea1817f14a7cac64c356048bdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:19.376410 27310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe5ea1817f14a7cac64c356048bdd62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:19.379109 27612 tablet_bootstrap.cc:492] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: Bootstrap starting.
I20260811 07:39:19.379452 27569 tablet_bootstrap.cc:492] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: Bootstrap starting.
I20260811 07:39:19.380259 27612 tablet_bootstrap.cc:654] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:19.380458 27569 tablet_bootstrap.cc:654] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:19.381286 27613 tablet_bootstrap.cc:492] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: Bootstrap starting.
I20260811 07:39:19.381522 27612 tablet_bootstrap.cc:492] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: No bootstrap required, opened a new log
I20260811 07:39:19.381559 27569 tablet_bootstrap.cc:492] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: No bootstrap required, opened a new log
I20260811 07:39:19.381628 27569 ts_tablet_manager.cc:1397] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:19.381624 27612 ts_tablet_manager.cc:1397] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:39:19.382066 27569 raft_consensus.cc:348] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.382066 27612 raft_consensus.cc:348] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.382192 27569 raft_consensus.cc:374] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:19.382192 27612 raft_consensus.cc:374] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:19.382236 27612 raft_consensus.cc:729] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5848513fbd48129664f5be11e46838, State: Initialized, Role: FOLLOWER
I20260811 07:39:19.382236 27569 raft_consensus.cc:729] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959392cac7cd490eadc2f337bd4d5e19, State: Initialized, Role: FOLLOWER
I20260811 07:39:19.382270 27613 tablet_bootstrap.cc:654] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:19.382368 27569 consensus_queue.cc:260] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.382392 27612 consensus_queue.cc:260] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.382617 27577 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.382727 27569 ts_tablet_manager.cc:1428] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:19.382755 27577 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:39:19.382841 27616 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.383042 27616 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:39:19.383750 27612 ts_tablet_manager.cc:1428] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:19.384632 27613 tablet_bootstrap.cc:492] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: No bootstrap required, opened a new log
I20260811 07:39:19.384716 27613 ts_tablet_manager.cc:1397] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 07:39:19.385157 27613 raft_consensus.cc:348] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.385250 27613 raft_consensus.cc:374] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:19.385272 27613 raft_consensus.cc:729] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be064db7576d4167a45e34ebcfe725d3, State: Initialized, Role: FOLLOWER
I20260811 07:39:19.385397 27613 consensus_queue.cc:260] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.385777 27613 ts_tablet_manager.cc:1428] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:19.385797 27618 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.385954 27618 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:39:19.634076 27618 raft_consensus.cc:482] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:19.634231 27618 raft_consensus.cc:504] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.635434 27618 leader_election.cc:288] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495), 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003)
I20260811 07:39:19.639801 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.636082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:58983 (local address 127.18.72.194:41003)
0811 07:39:19.636218 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.636225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.636242 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:39:19.636313 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.636318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.636372 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.636440 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:19.636854 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.637716 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.638808 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.639212 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.639354 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.639417 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.639595 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.639667 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:39:19.639839 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.635857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:38533 (local address 127.18.72.193:38495)
0811 07:39:19.636285 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.636288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.636302 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:19.636355 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.636357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.636395 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.636502 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:19.636873 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.637780 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.638818 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.639248 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.639401 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.639457 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.639630 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.639704 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":97,"threads_started":1}
I20260811 07:39:19.639859 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.635713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:38495 (local address 127.18.72.195:38533)
0811 07:39:19.636159 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:19.636175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:19.636282 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:19.636532 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:19.636536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:19.636567 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:19.636702 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.636713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.637909 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.637912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.638683 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.638692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.639391 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.639395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.639491 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.639519 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 07:39:19.639611 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:39:19.639682 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":70,"spinlock_wait_cycles":896,"thread_start_us":148,"threads_started":1}
I20260811 07:39:19.640133 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.635713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41003 (local address 127.18.72.195:58983)
0811 07:39:19.636007 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:19.636023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:19.636135 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:19.636492 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:19.636495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:19.636513 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:19.636707 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.636718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.637838 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.637841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.638673 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.638682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.639346 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.639350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.639460 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.639487 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 07:39:19.639576 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 07:39:19.639655 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":65,"thread_start_us":83,"threads_started":1}
I20260811 07:39:19.640460 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe5ea1817f14a7cac64c356048bdd62" candidate_uuid: "be064db7576d4167a45e34ebcfe725d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5848513fbd48129664f5be11e46838" is_pre_election: true
I20260811 07:39:19.640568 27330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe5ea1817f14a7cac64c356048bdd62" candidate_uuid: "be064db7576d4167a45e34ebcfe725d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959392cac7cd490eadc2f337bd4d5e19" is_pre_election: true
I20260811 07:39:19.640625 27183 raft_consensus.cc:2454] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be064db7576d4167a45e34ebcfe725d3 in term 0.
I20260811 07:39:19.640704 27330 raft_consensus.cc:2454] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be064db7576d4167a45e34ebcfe725d3 in term 0.
I20260811 07:39:19.641048 27412 leader_election.cc:302] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, be064db7576d4167a45e34ebcfe725d3; no voters:
I20260811 07:39:19.641342 27618 raft_consensus.cc:2781] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:19.641405 27618 raft_consensus.cc:482] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:19.641443 27618 raft_consensus.cc:3037] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:19.642452 27618 raft_consensus.cc:504] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.642799 27618 leader_election.cc:288] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [CANDIDATE]: Term 1 election: Requested vote from peers 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495), 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003)
I20260811 07:39:19.643133 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe5ea1817f14a7cac64c356048bdd62" candidate_uuid: "be064db7576d4167a45e34ebcfe725d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5848513fbd48129664f5be11e46838"
I20260811 07:39:19.643170 27330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe5ea1817f14a7cac64c356048bdd62" candidate_uuid: "be064db7576d4167a45e34ebcfe725d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959392cac7cd490eadc2f337bd4d5e19"
I20260811 07:39:19.643263 27183 raft_consensus.cc:3037] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:19.643306 27330 raft_consensus.cc:3037] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:19.644277 27183 raft_consensus.cc:2454] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be064db7576d4167a45e34ebcfe725d3 in term 1.
I20260811 07:39:19.644269 27330 raft_consensus.cc:2454] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be064db7576d4167a45e34ebcfe725d3 in term 1.
I20260811 07:39:19.644590 27412 leader_election.cc:302] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, be064db7576d4167a45e34ebcfe725d3; no voters:
I20260811 07:39:19.644764 27618 raft_consensus.cc:2781] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:19.645005 27618 raft_consensus.cc:686] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [term 1 LEADER]: Becoming Leader. State: Replica: be064db7576d4167a45e34ebcfe725d3, State: Running, Role: LEADER
I20260811 07:39:19.645269 27618 consensus_queue.cc:237] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.646525 27622 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: TxnStatusTablet state changed. Reason: New leader be064db7576d4167a45e34ebcfe725d3. Latest consensus state: current_term: 1 leader_uuid: "be064db7576d4167a45e34ebcfe725d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.646631 27622 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:39:19.646952 27624 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 07:39:19.647111 27624 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 07:39:19.647848 27011 catalog_manager.cc:5495] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to be064db7576d4167a45e34ebcfe725d3 (127.18.72.195). New cstate: current_term: 1 leader_uuid: "be064db7576d4167a45e34ebcfe725d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20260811 07:39:19.680089 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.675951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42529 (local address 127.0.0.1:45242)
0811 07:39:19.676138 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:19.676157 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:19.676276 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:19.676597 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:19.676600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:19.676606 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:39:19.676701 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.676705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.677704 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.677707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.678324 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.678332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.678906 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.678909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.678977 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.679024 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:39:19.679804 (+ 780us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:39:19.679816 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 07:39:19.679865 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 07:39:19.679981 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 07:39:19.680099 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.676135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45242 (local address 127.18.72.195:42529)
0811 07:39:19.676294 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.676299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.676315 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:39:19.676401 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.676405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.676469 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.676580 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:19.676809 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.677596 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.678442 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.678789 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.678901 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.679161 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:19.679704 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:19.679819 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.679884 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.679958 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:39:19.683288 27183 raft_consensus.cc:1261] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Refusing update from remote peer be064db7576d4167a45e34ebcfe725d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:19.683506 27330 raft_consensus.cc:1261] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Refusing update from remote peer be064db7576d4167a45e34ebcfe725d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:19.683806 27622 consensus_queue.cc:1035] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:19.684098 27618 consensus_queue.cc:1035] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:19.685631 27577 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: TxnStatusTablet state changed. Reason: New leader be064db7576d4167a45e34ebcfe725d3. Latest consensus state: current_term: 1 leader_uuid: "be064db7576d4167a45e34ebcfe725d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.685786 27577 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:39:19.685889 27616 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: TxnStatusTablet state changed. Reason: New leader be064db7576d4167a45e34ebcfe725d3. Latest consensus state: current_term: 1 leader_uuid: "be064db7576d4167a45e34ebcfe725d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.686002 27616 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:39:19.688596 27622 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be064db7576d4167a45e34ebcfe725d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.688714 27622 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:39:19.689034 27616 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be064db7576d4167a45e34ebcfe725d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.689134 27616 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P 959392cac7cd490eadc2f337bd4d5e19: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:39:19.689462 27629 mvcc.cc:205] Tried to move back new op lower bound from 7317233498857484288 to 7317233498709803008. Current Snapshot: MvccSnapshot[applied={T|T < 7317233498857484288}]
I20260811 07:39:19.692123 27622 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be064db7576d4167a45e34ebcfe725d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.692226 27622 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P be064db7576d4167a45e34ebcfe725d3: This TxnStatusTablet replica's current role is: LEADER
I20260811 07:39:19.692979 27577 tablet_replica.cc:439] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be064db7576d4167a45e34ebcfe725d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } } }
I20260811 07:39:19.693089 27577 tablet_replica.cc:442] T ebe5ea1817f14a7cac64c356048bdd62 P 0f5848513fbd48129664f5be11e46838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 07:39:19.700594 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.697027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.18.72.193:38495)
0811 07:39:19.697185 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.697192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.697205 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:19.697285 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.697288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.697336 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.697447 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:19.697692 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.698209 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.699063 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.699319 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.699427 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.699720 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:19.700224 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:19.700354 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.700414 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.700467 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:39:19.710969 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.707271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42529 (local address 127.18.72.193:55453)
0811 07:39:19.707554 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:19.707566 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:19.707649 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:19.707865 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:19.707868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:19.707873 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:39:19.708027 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.708031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.709036 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.709039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.709701 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.709711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.710316 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.710318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.710390 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.710431 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:39:19.710720 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:39:19.710726 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:39:19.710792 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 07:39:19.710851 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":99,"threads_started":1}
I20260811 07:39:19.710978 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.707371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:55453 (local address 127.18.72.195:42529)
0811 07:39:19.707488 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.707493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.707632 (+ 139us) server_negotiation.cc:407] Connection header received
0811 07:39:19.707719 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.707722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.707772 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.707857 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:19.708152 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.708913 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.709857 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.710193 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.710318 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.710559 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:19.710637 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:19.710706 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.710794 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.710851 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:39:19.718272 27488 raft_consensus.cc:1261] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Refusing update from remote peer 0f5848513fbd48129664f5be11e46838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:19.718272 27330 raft_consensus.cc:1261] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Refusing update from remote peer 0f5848513fbd48129664f5be11e46838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:19.718765 27577 consensus_queue.cc:1035] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:19.718967 27602 consensus_queue.cc:1035] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:19.741510 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.737665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:34937 (local address 127.18.72.193:38495)
0811 07:39:19.737811 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.737816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.738113 (+ 297us) server_negotiation.cc:407] Connection header received
0811 07:39:19.738164 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.738166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.738209 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.738277 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:19.738576 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.739292 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.740510 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.740712 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.740809 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.741108 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:19.741172 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:19.741224 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.741346 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.741387 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:39:19.741550 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.737601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:38495 (local address 127.18.72.195:34937)
0811 07:39:19.737968 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:19.737984 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:19.738091 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:19.738308 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:19.738310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:19.738318 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:39:19.738442 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.738448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.739408 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.739412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.740395 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:19.740407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.740819 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.740823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:19.740928 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.740982 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:39:19.741252 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:39:19.741259 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:39:19.741325 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 07:39:19.741404 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":133,"threads_started":1}
I20260811 07:39:19.769656 27010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:19.769879 27010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:19.772317 27010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: Sending DeleteTablet for 3 replicas of tablet e467f189efab4d6099c76a28ba54628c
I20260811 07:39:19.772864 27158 tablet_service.cc:1514] Processing DeleteTablet for tablet e467f189efab4d6099c76a28ba54628c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:19 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35110
I20260811 07:39:19.773137 27310 tablet_service.cc:1514] Processing DeleteTablet for tablet e467f189efab4d6099c76a28ba54628c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:19 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35386
I20260811 07:39:19.773593 27465 tablet_service.cc:1514] Processing DeleteTablet for tablet e467f189efab4d6099c76a28ba54628c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:19 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45240
I20260811 07:39:19.773978 27651 tablet_replica.cc:330] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: stopping tablet replica
I20260811 07:39:19.774120 27651 raft_consensus.cc:2229] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:19.774242 27651 raft_consensus.cc:2258] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:19.774479 27652 tablet_replica.cc:330] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: stopping tablet replica
I20260811 07:39:19.774595 27652 raft_consensus.cc:2229] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:19.774703 27652 raft_consensus.cc:2258] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:19.775435 27652 ts_tablet_manager.cc:1905] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:19.776721 27651 ts_tablet_manager.cc:1905] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:19.777757 27650 tablet_replica.cc:330] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: stopping tablet replica
I20260811 07:39:19.777937 27650 raft_consensus.cc:2229] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:39:19.778301 27650 raft_consensus.cc:2258] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:19.779392 27650 ts_tablet_manager.cc:1905] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:19.779908 27651 ts_tablet_manager.cc:1918] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:39:19.780005 27651 log.cc:1196] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/e467f189efab4d6099c76a28ba54628c
I20260811 07:39:19.780293 27651 ts_tablet_manager.cc:1939] T e467f189efab4d6099c76a28ba54628c P 959392cac7cd490eadc2f337bd4d5e19: Deleting consensus metadata
I20260811 07:39:19.781056 26997 catalog_manager.cc:4841] TS 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003): tablet e467f189efab4d6099c76a28ba54628c (table test-table [id=577ee887684c4ae09d83247a95fc5b5e]) successfully deleted
I20260811 07:39:19.783771 27652 ts_tablet_manager.cc:1918] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:39:19.783875 27652 log.cc:1196] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/e467f189efab4d6099c76a28ba54628c
I20260811 07:39:19.784250 27652 ts_tablet_manager.cc:1939] T e467f189efab4d6099c76a28ba54628c P be064db7576d4167a45e34ebcfe725d3: Deleting consensus metadata
I20260811 07:39:19.784813 27650 ts_tablet_manager.cc:1918] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:39:19.785009 26997 catalog_manager.cc:4841] TS be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529): tablet e467f189efab4d6099c76a28ba54628c (table test-table [id=577ee887684c4ae09d83247a95fc5b5e]) successfully deleted
I20260811 07:39:19.785177 27650 log.cc:1196] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/e467f189efab4d6099c76a28ba54628c
I20260811 07:39:19.785919 27650 ts_tablet_manager.cc:1939] T e467f189efab4d6099c76a28ba54628c P 0f5848513fbd48129664f5be11e46838: Deleting consensus metadata
I20260811 07:39:19.788422 26999 catalog_manager.cc:4841] TS 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495): tablet e467f189efab4d6099c76a28ba54628c (table test-table [id=577ee887684c4ae09d83247a95fc5b5e]) successfully deleted
Aug 11 07:39:19 dist-test-slave-zkpd krb5kdc[26965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 07:39:19 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:19 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 07:39:19.816735 27610 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:19.807956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.18.72.254:33387)
0811 07:39:19.808105 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.808109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.808707 (+ 598us) server_negotiation.cc:407] Connection header received
0811 07:39:19.808924 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.808928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.808978 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.809033 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:19.810123 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.810975 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.812389 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.812740 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.812815 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.816607 (+ 3792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59282: BlockingRecv error: recv got EOF from 127.0.0.1:59282 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 07:39:19 dist-test-slave-zkpd krb5kdc[26965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM
I20260811 07:39:19.828498 27610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:19.818747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.18.72.254:33387)
0811 07:39:19.818864 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:19.818868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:19.819060 (+ 192us) server_negotiation.cc:407] Connection header received
0811 07:39:19.819150 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:19.819153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:19.819195 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:19.819269 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:19.819968 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.820698 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.821781 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:19.822122 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:19.822236 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:19.824995 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:19.824999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:19.825001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:19.825027 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:19.826682 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:19.827146 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:19.827148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:19.827150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:19.827269 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:19.827589 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:19.827593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:19.827594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:19.827788 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:19.827958 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:19.828192 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 07:39:19.828299 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:39:19.831220 27010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:39:19.831691 27010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:19.836409 27158 tablet_service.cc:1467] Processing CreateTablet for tablet 9200af3ac7554113baa1a60d29203894 (DEFAULT_TABLE table=test-table [id=d8d02c0b9e884232b478e66590440365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:19.836632 27465 tablet_service.cc:1467] Processing CreateTablet for tablet 9200af3ac7554113baa1a60d29203894 (DEFAULT_TABLE table=test-table [id=d8d02c0b9e884232b478e66590440365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:19.836737 27158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9200af3ac7554113baa1a60d29203894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:19.836880 27465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9200af3ac7554113baa1a60d29203894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:19.838759 27613 tablet_bootstrap.cc:492] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: Bootstrap starting.
I20260811 07:39:19.839017 27310 tablet_service.cc:1467] Processing CreateTablet for tablet 9200af3ac7554113baa1a60d29203894 (DEFAULT_TABLE table=test-table [id=d8d02c0b9e884232b478e66590440365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:19.839150 27612 tablet_bootstrap.cc:492] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: Bootstrap starting.
I20260811 07:39:19.839293 27310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9200af3ac7554113baa1a60d29203894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:19.839632 27613 tablet_bootstrap.cc:654] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:19.840128 27612 tablet_bootstrap.cc:654] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:19.841482 27612 tablet_bootstrap.cc:492] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: No bootstrap required, opened a new log
I20260811 07:39:19.841539 27569 tablet_bootstrap.cc:492] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: Bootstrap starting.
I20260811 07:39:19.841564 27612 ts_tablet_manager.cc:1397] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:19.841981 27612 raft_consensus.cc:348] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.842087 27612 raft_consensus.cc:374] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:19.842118 27612 raft_consensus.cc:729] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5848513fbd48129664f5be11e46838, State: Initialized, Role: FOLLOWER
I20260811 07:39:19.842227 27612 consensus_queue.cc:260] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.842341 27569 tablet_bootstrap.cc:654] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:19.842494 27612 ts_tablet_manager.cc:1428] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:19.843597 27569 tablet_bootstrap.cc:492] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: No bootstrap required, opened a new log
I20260811 07:39:19.843691 27569 ts_tablet_manager.cc:1397] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:19.843823 27613 tablet_bootstrap.cc:492] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: No bootstrap required, opened a new log
I20260811 07:39:19.843920 27613 ts_tablet_manager.cc:1397] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 07:39:19.844166 27569 raft_consensus.cc:348] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.844271 27569 raft_consensus.cc:374] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:19.844306 27569 raft_consensus.cc:729] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959392cac7cd490eadc2f337bd4d5e19, State: Initialized, Role: FOLLOWER
I20260811 07:39:19.844293 27613 raft_consensus.cc:348] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.844393 27613 raft_consensus.cc:374] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:19.844424 27613 raft_consensus.cc:729] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be064db7576d4167a45e34ebcfe725d3, State: Initialized, Role: FOLLOWER
I20260811 07:39:19.844426 27569 consensus_queue.cc:260] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.844538 27613 consensus_queue.cc:260] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:19.844704 27569 ts_tablet_manager.cc:1428] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:19.844791 27613 ts_tablet_manager.cc:1428] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:20.072813 27616 raft_consensus.cc:482] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:20.072954 27616 raft_consensus.cc:504] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.073441 27616 leader_election.cc:288] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495), be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
I20260811 07:39:20.073980 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9200af3ac7554113baa1a60d29203894" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be064db7576d4167a45e34ebcfe725d3" is_pre_election: true
I20260811 07:39:20.073980 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9200af3ac7554113baa1a60d29203894" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5848513fbd48129664f5be11e46838" is_pre_election: true
I20260811 07:39:20.074138 27488 raft_consensus.cc:2454] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 0.
I20260811 07:39:20.074138 27183 raft_consensus.cc:2454] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 0.
I20260811 07:39:20.074517 27263 leader_election.cc:302] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, 959392cac7cd490eadc2f337bd4d5e19, be064db7576d4167a45e34ebcfe725d3; no voters:
I20260811 07:39:20.074805 27616 raft_consensus.cc:2781] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:20.074877 27616 raft_consensus.cc:482] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:20.074927 27616 raft_consensus.cc:3037] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:20.076340 27616 raft_consensus.cc:504] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.076723 27616 leader_election.cc:288] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 election: Requested vote from peers 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495), be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
I20260811 07:39:20.077087 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9200af3ac7554113baa1a60d29203894" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be064db7576d4167a45e34ebcfe725d3"
I20260811 07:39:20.077086 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9200af3ac7554113baa1a60d29203894" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5848513fbd48129664f5be11e46838"
I20260811 07:39:20.077217 27183 raft_consensus.cc:3037] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:20.077225 27488 raft_consensus.cc:3037] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:20.078382 27183 raft_consensus.cc:2454] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 1.
I20260811 07:39:20.078372 27488 raft_consensus.cc:2454] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 1.
I20260811 07:39:20.078687 27263 leader_election.cc:302] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, 959392cac7cd490eadc2f337bd4d5e19; no voters:
I20260811 07:39:20.078866 27616 raft_consensus.cc:2781] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:20.079015 27616 raft_consensus.cc:686] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 LEADER]: Becoming Leader. State: Replica: 959392cac7cd490eadc2f337bd4d5e19, State: Running, Role: LEADER
I20260811 07:39:20.079147 27616 consensus_queue.cc:237] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.080890 27010 catalog_manager.cc:5495] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194). New cstate: current_term: 1 leader_uuid: "959392cac7cd490eadc2f337bd4d5e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20260811 07:39:20.139956 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:20.137016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35406 (local address 127.18.72.194:41003)
0811 07:39:20.137159 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:20.137162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:20.137176 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:20.137224 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:20.137227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:20.137270 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:20.137336 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:20.137565 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.138071 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.138873 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.139080 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.139155 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:20.139395 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:20.139464 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:20.139519 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:20.139691 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 07:39:20.139739 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 07:39:20.142068 27183 raft_consensus.cc:1261] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Refusing update from remote peer 959392cac7cd490eadc2f337bd4d5e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:20.142423 27488 raft_consensus.cc:1261] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Refusing update from remote peer 959392cac7cd490eadc2f337bd4d5e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:20.142570 27616 consensus_queue.cc:1035] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:20.142925 27670 consensus_queue.cc:1035] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:20.149250 27012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:20.149410 27012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:20.151316 27012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: Sending DeleteTablet for 3 replicas of tablet 9200af3ac7554113baa1a60d29203894
I20260811 07:39:20.151942 27158 tablet_service.cc:1514] Processing DeleteTablet for tablet 9200af3ac7554113baa1a60d29203894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:20 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35110
I20260811 07:39:20.151969 27465 tablet_service.cc:1514] Processing DeleteTablet for tablet 9200af3ac7554113baa1a60d29203894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:20 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45240
I20260811 07:39:20.152122 27650 tablet_replica.cc:330] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: stopping tablet replica
I20260811 07:39:20.152123 27310 tablet_service.cc:1514] Processing DeleteTablet for tablet 9200af3ac7554113baa1a60d29203894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:20 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35386
I20260811 07:39:20.152212 27650 raft_consensus.cc:2229] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:20.152318 27651 tablet_replica.cc:330] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: stopping tablet replica
I20260811 07:39:20.152345 27650 raft_consensus.cc:2258] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:20.152410 27652 tablet_replica.cc:330] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: stopping tablet replica
I20260811 07:39:20.152441 27651 raft_consensus.cc:2229] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:39:20.152532 27652 raft_consensus.cc:2229] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:20.152614 27651 raft_consensus.cc:2258] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:20.152657 27652 raft_consensus.cc:2258] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:20.152688 27012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 07:39:20.153023 27650 ts_tablet_manager.cc:1905] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 07:39:20.153142 27012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:20.153633 27652 ts_tablet_manager.cc:1905] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:20.154261 27651 ts_tablet_manager.cc:1905] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:20.155573 27650 ts_tablet_manager.cc:1918] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:20.155673 27650 log.cc:1196] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/9200af3ac7554113baa1a60d29203894
I20260811 07:39:20.155975 27650 ts_tablet_manager.cc:1939] T 9200af3ac7554113baa1a60d29203894 P 0f5848513fbd48129664f5be11e46838: Deleting consensus metadata
I20260811 07:39:20.156257 27652 ts_tablet_manager.cc:1918] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:20.156347 27652 log.cc:1196] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/9200af3ac7554113baa1a60d29203894
I20260811 07:39:20.156613 27652 ts_tablet_manager.cc:1939] T 9200af3ac7554113baa1a60d29203894 P be064db7576d4167a45e34ebcfe725d3: Deleting consensus metadata
I20260811 07:39:20.156694 26999 catalog_manager.cc:4841] TS 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495): tablet 9200af3ac7554113baa1a60d29203894 (table test-table [id=d8d02c0b9e884232b478e66590440365]) successfully deleted
I20260811 07:39:20.156958 27651 ts_tablet_manager.cc:1918] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:20.157042 27651 log.cc:1196] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/9200af3ac7554113baa1a60d29203894
I20260811 07:39:20.157316 27651 ts_tablet_manager.cc:1939] T 9200af3ac7554113baa1a60d29203894 P 959392cac7cd490eadc2f337bd4d5e19: Deleting consensus metadata
I20260811 07:39:20.157294 26997 catalog_manager.cc:4841] TS be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529): tablet 9200af3ac7554113baa1a60d29203894 (table test-table [id=d8d02c0b9e884232b478e66590440365]) successfully deleted
I20260811 07:39:20.157903 26997 catalog_manager.cc:4841] TS 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003): tablet 9200af3ac7554113baa1a60d29203894 (table test-table [id=d8d02c0b9e884232b478e66590440365]) successfully deleted
I20260811 07:39:20.160540 27158 tablet_service.cc:1467] Processing CreateTablet for tablet 4c49e141f7984d49ad0c9a4797ae34f7 (DEFAULT_TABLE table=test-table [id=aebf40736cee4bcf8883289587f92a59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:20.160637 27310 tablet_service.cc:1467] Processing CreateTablet for tablet 4c49e141f7984d49ad0c9a4797ae34f7 (DEFAULT_TABLE table=test-table [id=aebf40736cee4bcf8883289587f92a59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:20.160674 27465 tablet_service.cc:1467] Processing CreateTablet for tablet 4c49e141f7984d49ad0c9a4797ae34f7 (DEFAULT_TABLE table=test-table [id=aebf40736cee4bcf8883289587f92a59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:20.160822 27158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c49e141f7984d49ad0c9a4797ae34f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:20.160885 27310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c49e141f7984d49ad0c9a4797ae34f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:20.160892 27465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c49e141f7984d49ad0c9a4797ae34f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:20.162575 27612 tablet_bootstrap.cc:492] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838: Bootstrap starting.
I20260811 07:39:20.163057 27569 tablet_bootstrap.cc:492] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: Bootstrap starting.
I20260811 07:39:20.163137 27613 tablet_bootstrap.cc:492] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: Bootstrap starting.
I20260811 07:39:20.163443 27612 tablet_bootstrap.cc:654] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:20.163830 27569 tablet_bootstrap.cc:654] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:20.163833 27613 tablet_bootstrap.cc:654] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:20.164788 27612 tablet_bootstrap.cc:492] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838: No bootstrap required, opened a new log
I20260811 07:39:20.164906 27612 ts_tablet_manager.cc:1397] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:20.164942 27569 tablet_bootstrap.cc:492] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: No bootstrap required, opened a new log
I20260811 07:39:20.165019 27569 ts_tablet_manager.cc:1397] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:20.165179 27613 tablet_bootstrap.cc:492] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: No bootstrap required, opened a new log
I20260811 07:39:20.165261 27613 ts_tablet_manager.cc:1397] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:20.165393 27612 raft_consensus.cc:348] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.165509 27612 raft_consensus.cc:374] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:20.165452 27569 raft_consensus.cc:348] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.165550 27612 raft_consensus.cc:729] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5848513fbd48129664f5be11e46838, State: Initialized, Role: FOLLOWER
I20260811 07:39:20.165555 27569 raft_consensus.cc:374] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:20.165591 27569 raft_consensus.cc:729] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959392cac7cd490eadc2f337bd4d5e19, State: Initialized, Role: FOLLOWER
I20260811 07:39:20.165633 27613 raft_consensus.cc:348] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.165736 27613 raft_consensus.cc:374] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:20.165686 27612 consensus_queue.cc:260] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.165736 27569 consensus_queue.cc:260] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.165793 27613 raft_consensus.cc:729] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be064db7576d4167a45e34ebcfe725d3, State: Initialized, Role: FOLLOWER
I20260811 07:39:20.165915 27613 consensus_queue.cc:260] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.165989 27612 ts_tablet_manager.cc:1428] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:20.166023 27569 ts_tablet_manager.cc:1428] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:20.166172 27613 ts_tablet_manager.cc:1428] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:39:20.226883 27616 raft_consensus.cc:482] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:20.227015 27616 raft_consensus.cc:504] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.227423 27616 leader_election.cc:288] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495), be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
I20260811 07:39:20.227916 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c49e141f7984d49ad0c9a4797ae34f7" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5848513fbd48129664f5be11e46838" is_pre_election: true
I20260811 07:39:20.227908 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c49e141f7984d49ad0c9a4797ae34f7" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be064db7576d4167a45e34ebcfe725d3" is_pre_election: true
I20260811 07:39:20.228076 27183 raft_consensus.cc:2454] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 0.
I20260811 07:39:20.228087 27488 raft_consensus.cc:2454] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 0.
I20260811 07:39:20.228397 27263 leader_election.cc:302] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5848513fbd48129664f5be11e46838, 959392cac7cd490eadc2f337bd4d5e19; no voters:
I20260811 07:39:20.228613 27616 raft_consensus.cc:2781] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:20.228694 27616 raft_consensus.cc:482] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:20.228749 27616 raft_consensus.cc:3037] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:20.229772 27616 raft_consensus.cc:504] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.230142 27616 leader_election.cc:288] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 election: Requested vote from peers 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495), be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529)
I20260811 07:39:20.230510 27183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c49e141f7984d49ad0c9a4797ae34f7" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5848513fbd48129664f5be11e46838"
I20260811 07:39:20.230509 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c49e141f7984d49ad0c9a4797ae34f7" candidate_uuid: "959392cac7cd490eadc2f337bd4d5e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be064db7576d4167a45e34ebcfe725d3"
I20260811 07:39:20.230643 27488 raft_consensus.cc:3037] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:20.230643 27183 raft_consensus.cc:3037] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:20.231787 27183 raft_consensus.cc:2454] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 1.
I20260811 07:39:20.231787 27488 raft_consensus.cc:2454] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 959392cac7cd490eadc2f337bd4d5e19 in term 1.
I20260811 07:39:20.232120 27261 leader_election.cc:302] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959392cac7cd490eadc2f337bd4d5e19, be064db7576d4167a45e34ebcfe725d3; no voters:
I20260811 07:39:20.232342 27616 raft_consensus.cc:2781] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:20.232443 27616 raft_consensus.cc:686] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 LEADER]: Becoming Leader. State: Replica: 959392cac7cd490eadc2f337bd4d5e19, State: Running, Role: LEADER
I20260811 07:39:20.232604 27616 consensus_queue.cc:237] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } }
I20260811 07:39:20.234167 27010 catalog_manager.cc:5495] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194). New cstate: current_term: 1 leader_uuid: "959392cac7cd490eadc2f337bd4d5e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959392cac7cd490eadc2f337bd4d5e19" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20260811 07:39:20.258723 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:20.255247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42529 (local address 127.18.72.194:53023)
0811 07:39:20.255513 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:20.255531 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:20.255642 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:20.255970 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:20.255973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:20.255981 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:39:20.256096 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:20.256102 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.256844 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.256848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:20.257468 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:20.257476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.257944 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.257948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:20.258041 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:20.258086 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:39:20.258395 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:39:20.258404 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 07:39:20.258494 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 07:39:20.258581 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260811 07:39:20.258730 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:20.255352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:53023 (local address 127.18.72.195:42529)
0811 07:39:20.255579 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:20.255584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:20.255598 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:20.255777 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:20.255784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:20.255831 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:20.255958 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:20.256208 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.256731 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.257592 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.257831 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.257941 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:20.258234 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:20.258308 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:20.258388 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:20.258507 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 07:39:20.258563 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":53,"threads_started":1}
I20260811 07:39:20.264452 27488 raft_consensus.cc:1261] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Refusing update from remote peer 959392cac7cd490eadc2f337bd4d5e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:20.264520 27183 raft_consensus.cc:1261] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Refusing update from remote peer 959392cac7cd490eadc2f337bd4d5e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:20.264878 27670 consensus_queue.cc:1035] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be064db7576d4167a45e34ebcfe725d3" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:20.265008 27616 consensus_queue.cc:1035] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f5848513fbd48129664f5be11e46838" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:20.281128 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:20.277805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:55541 (local address 127.18.72.194:41003)
0811 07:39:20.277958 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:20.277963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:20.278040 (+ 77us) server_negotiation.cc:407] Connection header received
0811 07:39:20.278193 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:20.278202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:20.278253 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:20.278334 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:20.278572 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.279080 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.280015 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.280314 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.280432 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:20.280673 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:20.280758 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:20.280848 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:20.280922 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 07:39:20.280983 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 07:39:20.281177 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:20.277709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41003 (local address 127.18.72.195:55541)
0811 07:39:20.277970 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:20.277984 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:20.278085 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:20.278345 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:20.278348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:20.278358 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 07:39:20.278461 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:20.278466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.279192 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.279196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:20.279883 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:20.279917 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:20.280437 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:20.280440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:20.280526 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:20.280555 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 07:39:20.280836 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 07:39:20.280843 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:39:20.280910 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 07:39:20.281034 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260811 07:39:20.301631 27011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:20.301828 27011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:20.303843 27011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e08455968ef409ba779bf717cab75d2: Sending DeleteTablet for 3 replicas of tablet 4c49e141f7984d49ad0c9a4797ae34f7
I20260811 07:39:20.304440 27158 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c49e141f7984d49ad0c9a4797ae34f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:20 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35110
I20260811 07:39:20.304559 27465 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c49e141f7984d49ad0c9a4797ae34f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:20 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45240
I20260811 07:39:20.304580 27310 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c49e141f7984d49ad0c9a4797ae34f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:20 UTC) from {username='oryx', principal='oryx/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35386
I20260811 07:39:20.304639 27650 tablet_replica.cc:330] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838: stopping tablet replica
I20260811 07:39:20.304771 27651 tablet_replica.cc:330] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: stopping tablet replica
I20260811 07:39:20.304766 27650 raft_consensus.cc:2229] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:20.304873 27651 raft_consensus.cc:2229] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:39:20.304889 27650 raft_consensus.cc:2258] T 4c49e141f7984d49ad0c9a4797ae34f7 P 0f5848513fbd48129664f5be11e46838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:20.305059 27652 tablet_replica.cc:330] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: stopping tablet replica
I20260811 07:39:20.305050 27651 raft_consensus.cc:2258] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:20.305187 27652 raft_consensus.cc:2229] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:20.305231 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 27089
I20260811 07:39:20.305295 27652 raft_consensus.cc:2258] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:20.306499 27651 ts_tablet_manager.cc:1905] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:20.308418 27652 ts_tablet_manager.cc:1905] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:20.310060 27651 ts_tablet_manager.cc:1918] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:39:20.310161 27651 log.cc:1196] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/4c49e141f7984d49ad0c9a4797ae34f7
I20260811 07:39:20.310460 27651 ts_tablet_manager.cc:1939] T 4c49e141f7984d49ad0c9a4797ae34f7 P 959392cac7cd490eadc2f337bd4d5e19: Deleting consensus metadata
I20260811 07:39:20.310961 27652 ts_tablet_manager.cc:1918] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 07:39:20.311057 27652 log.cc:1196] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/4c49e141f7984d49ad0c9a4797ae34f7
I20260811 07:39:20.311187 26997 catalog_manager.cc:4841] TS 959392cac7cd490eadc2f337bd4d5e19 (127.18.72.194:41003): tablet 4c49e141f7984d49ad0c9a4797ae34f7 (table test-table [id=aebf40736cee4bcf8883289587f92a59]) successfully deleted
I20260811 07:39:20.311331 27652 ts_tablet_manager.cc:1939] T 4c49e141f7984d49ad0c9a4797ae34f7 P be064db7576d4167a45e34ebcfe725d3: Deleting consensus metadata
I20260811 07:39:20.311976 26997 catalog_manager.cc:4841] TS be064db7576d4167a45e34ebcfe725d3 (127.18.72.195:42529): tablet 4c49e141f7984d49ad0c9a4797ae34f7 (table test-table [id=aebf40736cee4bcf8883289587f92a59]) successfully deleted
W20260811 07:39:20.318712 26999 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:38495 (error 108)
I20260811 07:39:20.319100 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 27243
I20260811 07:39:20.321861 27692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:20.320631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:38495 (local address 127.0.0.1:35132)
0811 07:39:20.321707 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:20.321776 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1003,"thread_start_us":58,"threads_started":1}
W20260811 07:39:20.322041 26999 catalog_manager.cc:4584] TS 0f5848513fbd48129664f5be11e46838 (127.18.72.193:38495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c49e141f7984d49ad0c9a4797ae34f7: Network error: Client connection negotiation failed: client connection to 127.18.72.193:38495: connect: Connection refused (error 111)
I20260811 07:39:20.329108 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 27391
I20260811 07:39:20.337895 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 26979
2026-08-11T07:39:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3950 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:20 dist-test-slave-zkpd krb5kdc[27695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:20 dist-test-slave-zkpd krb5kdc[27695](info): set up 2 sockets
Aug 11 07:39:20 dist-test-slave-zkpd krb5kdc[27695](info): commencing operation
krb5kdc: starting...
W20260811 07:39:21.437215 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:21 dist-test-slave-zkpd krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:39:21.564821 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:40791
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:40791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:21.684969 27709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:21.685220 27709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:21.685258 27709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:21.689232 27709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:21.689287 27709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:21.689306 27709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:21.689322 27709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:21.689355 27709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:21.694976 27709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:40791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:40791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:21.696102 27709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:21.697170 27709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:21.706429 27721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:21.706534 27716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:21.706755 27709 server_base.cc:1034] running on GCE node
W20260811 07:39:21.706429 27718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:21.707588 27709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:21.708828 27709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:21.710021 27709 hybrid_clock.cc:648] HybridClock initialized: now 1786433961709991 us; error 43 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T07:39:21Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1382 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:21 dist-test-slave-zkpd krb5kdc[27726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:21 dist-test-slave-zkpd krb5kdc[27726](info): set up 2 sockets
Aug 11 07:39:21 dist-test-slave-zkpd krb5kdc[27726](info): commencing operation
krb5kdc: starting...
W20260811 07:39:22.803944 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:22 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:39:22.934696 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:36435
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:36435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:23.047111 27739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:23.047437 27739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:23.047523 27739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:23.051514 27739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:23.051601 27739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:23.051640 27739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:23.051699 27739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:23.051738 27739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:23.057003 27739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:36435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:36435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:23.058059 27739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:23.059120 27739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:23.067564 27747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:23.067600 27745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:23.067585 27744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:23.067677 27739 server_base.cc:1034] running on GCE node
I20260811 07:39:23.068450 27739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:23.069548 27739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:23.070706 27739 hybrid_clock.cc:648] HybridClock initialized: now 1786433963070676 us; error 43 us; skew 500 ppm
Aug 11 07:39:23 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:23.073518 27739 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:39:23.074378 27739 webserver.cc:460] Webserver started at http://127.18.72.254:36219/ using document root <none> and password file <none>
I20260811 07:39:23.074950 27739 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:23.075013 27739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:23.075270 27739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:23.077364 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "30d11aeb9e944ed48e5720e89f5292bb"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "d277c42e854f5266f4746df14a54955a"
server_key_iv: "162f479883ebffb3358890e4e7729538"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.077946 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "30d11aeb9e944ed48e5720e89f5292bb"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "d277c42e854f5266f4746df14a54955a"
server_key_iv: "162f479883ebffb3358890e4e7729538"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.081753 27739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:39:23.084101 27754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.085301 27739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:23.085400 27739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "30d11aeb9e944ed48e5720e89f5292bb"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "d277c42e854f5266f4746df14a54955a"
server_key_iv: "162f479883ebffb3358890e4e7729538"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.085480 27739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:23.113008 27739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:23.115772 27739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:23.116003 27739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:23.123215 27739 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:36435
I20260811 07:39:23.123245 27811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:36435 every 8 connection(s)
I20260811 07:39:23.124220 27739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:23.127051 27812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:23.131254 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 27739
I20260811 07:39:23.131407 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:23.131697 18723 external_mini_cluster.cc:1428] Setting key f85dee04af65784cde5e47db607ebf70
I20260811 07:39:23.133126 27812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Bootstrap starting.
I20260811 07:39:23.135314 27812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:23.136106 27812 log.cc:824] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:23.137849 27812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: No bootstrap required, opened a new log
Aug 11 07:39:23 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:23.141964 27812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } }
I20260811 07:39:23.142210 27812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:23.142294 27812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d11aeb9e944ed48e5720e89f5292bb, State: Initialized, Role: FOLLOWER
I20260811 07:39:23.142755 27812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } }
I20260811 07:39:23.142885 27812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:23.142997 27812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:23.143131 27812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:23.144019 27812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } }
I20260811 07:39:23.144351 27812 leader_election.cc:302] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30d11aeb9e944ed48e5720e89f5292bb; no voters:
I20260811 07:39:23.144677 27812 leader_election.cc:288] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:23.144745 27818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:23.145290 27818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 1 LEADER]: Becoming Leader. State: Replica: 30d11aeb9e944ed48e5720e89f5292bb, State: Running, Role: LEADER
I20260811 07:39:23.145601 27818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } }
I20260811 07:39:23.146104 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:23.133119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38122 (local address 127.18.72.254:36435)
0811 07:39:23.133547 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:23.133563 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:23.133597 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:39:23.134221 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:23.134237 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:23.134563 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:23.134894 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:23.135690 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.136781 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.137645 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.138147 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.138309 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:23.141050 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:23.141055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:23.141068 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:23.141092 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:23.142878 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:23.143387 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:23.143393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:23.143399 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:23.143513 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:23.143834 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:23.143837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:23.143839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:23.144355 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:23.144558 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:23.144856 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 07:39:23.145191 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":140,"threads_started":1}
I20260811 07:39:23.146122 27812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:23.146701 27818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30d11aeb9e944ed48e5720e89f5292bb. Latest consensus state: current_term: 1 leader_uuid: "30d11aeb9e944ed48e5720e89f5292bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } } }
I20260811 07:39:23.146917 27818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:23.147128 27821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30d11aeb9e944ed48e5720e89f5292bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } } }
I20260811 07:39:23.147287 27821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:23.147406 27828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:23.150524 27828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:23.156384 27828 catalog_manager.cc:1347] Generated new cluster ID: de3ae1dae72542268682e5a37c94229d
I20260811 07:39:23.156454 27828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:23.160714 27828 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:23.161741 27828 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:23.168375 27828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Generated new TSK 0
I20260811 07:39:23.168993 27828 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:39:23.204694 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36435
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:23.342223 27841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:23.342468 27841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:23.342510 27841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:23.346451 27841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:23.346532 27841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:23.346617 27841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:23.351835 27841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:23.352885 27841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:23.353945 27841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:23.363552 27849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:23.363754 27841 server_base.cc:1034] running on GCE node
W20260811 07:39:23.363561 27846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:23.363561 27847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:23.364290 27841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:23.364977 27841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:23.366190 27841 hybrid_clock.cc:648] HybridClock initialized: now 1786433963366169 us; error 28 us; skew 500 ppm
Aug 11 07:39:23 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:23.369361 27841 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:39:23.370201 27841 webserver.cc:460] Webserver started at http://127.18.72.193:42659/ using document root <none> and password file <none>
I20260811 07:39:23.370757 27841 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:23.370820 27841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:23.370990 27841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:23.373122 27841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "d1505d4263bd471c956b17a9e64dc163"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "1b83c5b376c0b1d93f0110aea6723829"
server_key_iv: "3b72835f0d7aa91ec74d2e91914a42ba"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.373634 27841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "d1505d4263bd471c956b17a9e64dc163"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "1b83c5b376c0b1d93f0110aea6723829"
server_key_iv: "3b72835f0d7aa91ec74d2e91914a42ba"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.377585 27841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:39:23.380226 27858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.381579 27841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:23.381709 27841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "d1505d4263bd471c956b17a9e64dc163"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "1b83c5b376c0b1d93f0110aea6723829"
server_key_iv: "3b72835f0d7aa91ec74d2e91914a42ba"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.381825 27841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:23.399587 27841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:23.403147 27841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:23.403316 27841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:23.404034 27841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:23.404889 27841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:23.404942 27841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.405012 27841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:23.405042 27841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.413677 27841 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:43037
I20260811 07:39:23.413774 27978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:43037 every 8 connection(s)
I20260811 07:39:23.414656 27841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:23.422250 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 27841
I20260811 07:39:23.422391 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:23.422703 18723 external_mini_cluster.cc:1428] Setting key 31a9ef995cea9bf3152b3a848c581203
Aug 11 07:39:23 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:23.428475 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:23.416530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:53337 (local address 127.18.72.254:36435)
0811 07:39:23.416701 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:23.416706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:23.417729 (+ 1023us) server_negotiation.cc:407] Connection header received
0811 07:39:23.418567 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:23.418569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:23.418615 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:23.418688 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:23.419853 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.420460 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.421545 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.421762 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.421855 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:23.424693 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:23.424698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:23.424701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:23.424727 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:23.426186 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:23.426759 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:23.426765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:23.426769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:23.426864 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:23.427188 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:23.427194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:23.427197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:23.427446 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:23.427558 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:23.428095 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 07:39:23.428235 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 07:39:23.429189 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:23.416907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:53337)
0811 07:39:23.417578 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:23.417619 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:23.418384 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:23.418856 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:23.418862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:23.419204 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:23.419698 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:23.419711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.420575 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.420577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:23.421441 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:23.421451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.421905 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.421907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:23.422048 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:23.422698 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:23.422714 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:23.424504 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:23.426310 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:23.426319 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:23.426339 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:23.426659 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:23.426961 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:23.426964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:23.426965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:23.427096 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:23.427555 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:23.427560 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:23.427802 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 07:39:23.428035 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 07:39:23.428292 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":115,"threads_started":1}
I20260811 07:39:23.430440 27979 heartbeater.cc:347] Connected to a master server at 127.18.72.254:36435
I20260811 07:39:23.430721 27979 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:23.431380 27979 heartbeater.cc:511] Master 127.18.72.254:36435 requested a full tablet report, sending...
I20260811 07:39:23.433270 27773 ts_manager.cc:194] Registered new tserver with Master: d1505d4263bd471c956b17a9e64dc163 (127.18.72.193:43037)
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
I20260811 07:39:23.434900 27773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:53337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:39:23.455554 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36435
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:23.572584 27984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:23.572846 27984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:23.572925 27984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:23.576897 27984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:23.576963 27984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:23.577021 27984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:23.582262 27984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:23.583338 27984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:23.584498 27984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:23.594535 27989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:23.594535 27990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:23.594673 27984 server_base.cc:1034] running on GCE node
W20260811 07:39:23.594560 27993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:23.595317 27984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:23.596048 27984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:23.597232 27984 hybrid_clock.cc:648] HybridClock initialized: now 1786433963597199 us; error 42 us; skew 500 ppm
Aug 11 07:39:23 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:23.600100 27984 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:39:23.600953 27984 webserver.cc:460] Webserver started at http://127.18.72.194:35485/ using document root <none> and password file <none>
I20260811 07:39:23.601529 27984 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:23.601621 27984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:23.601858 27984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:23.604034 27984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "7bef2a92ed5143a58d22e465c05b4297"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "24f52610dd082ea190cf6b30af1c791f"
server_key_iv: "8e3e35d29a49f46f3b438746ad6b7396"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.604605 27984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "7bef2a92ed5143a58d22e465c05b4297"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "24f52610dd082ea190cf6b30af1c791f"
server_key_iv: "8e3e35d29a49f46f3b438746ad6b7396"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.608249 27984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:39:23.610647 28002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.611784 27984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:23.611939 27984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "7bef2a92ed5143a58d22e465c05b4297"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "24f52610dd082ea190cf6b30af1c791f"
server_key_iv: "8e3e35d29a49f46f3b438746ad6b7396"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.612058 27984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:23.631407 27984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:23.634460 27984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:23.634665 27984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:23.635270 27984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:23.636173 27984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:23.636240 27984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.636319 27984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:23.636369 27984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.645605 27984 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:46429
I20260811 07:39:23.645677 28124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:46429 every 8 connection(s)
I20260811 07:39:23.646597 27984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:23.653258 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 27984
I20260811 07:39:23.653386 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:23.653669 18723 external_mini_cluster.cc:1428] Setting key 0edf0c3af722048bbae5411a85365335
Aug 11 07:39:23 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:23.661398 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:23.648605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:56653 (local address 127.18.72.254:36435)
0811 07:39:23.648758 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:23.648763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:23.649887 (+ 1124us) server_negotiation.cc:407] Connection header received
0811 07:39:23.650789 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:23.650791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:23.650846 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:23.650920 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:23.652207 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.652754 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.654038 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.654314 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.654419 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:23.657373 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:23.657380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:23.657383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:23.657416 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:23.659080 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:23.659569 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:23.659575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:23.659580 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:23.659672 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:23.660042 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:23.660048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:23.660053 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:23.660360 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:23.660495 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:23.661059 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 07:39:23.661192 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:39:23.662096 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:23.649058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:56653)
0811 07:39:23.649732 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:23.649773 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:23.650614 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:23.651143 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:23.651153 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:23.651516 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:23.652083 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:23.652098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.652858 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.652861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:23.653899 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:23.653910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.654478 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.654483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:23.654676 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:23.655363 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:23.655380 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:23.657170 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:23.659201 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:23.659205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:23.659217 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:23.659468 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:23.659764 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:23.659766 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:23.659767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:23.659951 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:23.660476 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:23.660482 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:23.660749 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 07:39:23.660940 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 07:39:23.661198 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":108,"threads_started":1}
I20260811 07:39:23.663414 28125 heartbeater.cc:347] Connected to a master server at 127.18.72.254:36435
I20260811 07:39:23.663709 28125 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:23.664397 28125 heartbeater.cc:511] Master 127.18.72.254:36435 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:39:23.665624 27773 ts_manager.cc:194] Registered new tserver with Master: 7bef2a92ed5143a58d22e465c05b4297 (127.18.72.194:46429)
I20260811 07:39:23.666213 27773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:56653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:39:23.684099 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36435
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:23.807070 28130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:23.807380 28130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:23.807464 28130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:23.811798 28130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:23.811880 28130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:23.812069 28130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:23.817924 28130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:23.819186 28130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:23.820420 28130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:23.830575 28136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:23.830607 28135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:23.830577 28140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:23.830958 28130 server_base.cc:1034] running on GCE node
I20260811 07:39:23.831564 28130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:23.832335 28130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:23.833539 28130 hybrid_clock.cc:648] HybridClock initialized: now 1786433963833518 us; error 29 us; skew 500 ppm
Aug 11 07:39:23 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:23.836635 28130 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:39:23.837603 28130 webserver.cc:460] Webserver started at http://127.18.72.195:39713/ using document root <none> and password file <none>
I20260811 07:39:23.838205 28130 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:23.838331 28130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:23.838577 28130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:23.840852 28130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "496de97553804ff8be25d3395c4d12e8"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "04f1d7c8a108f7f8df3d0a9cb0944233"
server_key_iv: "29923a0336646e522674d3c8dff72f40"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.841503 28130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "496de97553804ff8be25d3395c4d12e8"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "04f1d7c8a108f7f8df3d0a9cb0944233"
server_key_iv: "29923a0336646e522674d3c8dff72f40"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.845660 28130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:39:23.848589 28149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.849949 28130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:23.850098 28130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "496de97553804ff8be25d3395c4d12e8"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "04f1d7c8a108f7f8df3d0a9cb0944233"
server_key_iv: "29923a0336646e522674d3c8dff72f40"
server_key_version: "encryptionkey@0"
I20260811 07:39:23.850221 28130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:23.879293 28130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:23.883359 28130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:23.883606 28130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:23.884371 28130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:23.885330 28130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:23.885383 28130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.885463 28130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:23.885504 28130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:23.895635 28130 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:45707
I20260811 07:39:23.895656 28271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:45707 every 8 connection(s)
I20260811 07:39:23.896723 28130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:23.901656 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 28130
I20260811 07:39:23.901821 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:23.902119 18723 external_mini_cluster.cc:1428] Setting key 2edbfde28b22ddd2f51720b69abe6819
Aug 11 07:39:23 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:23.911055 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:23.898769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:51081 (local address 127.18.72.254:36435)
0811 07:39:23.898952 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:23.898956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:23.899818 (+ 862us) server_negotiation.cc:407] Connection header received
0811 07:39:23.901208 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:23.901212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:23.901269 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:23.901388 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:23.902872 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.903587 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.904907 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.905132 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.905244 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:23.907618 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:23.907622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:23.907625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:23.907660 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:23.908979 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:23.909499 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:23.909503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:23.909505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:23.909592 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:23.909949 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:23.909953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:23.909955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:23.910143 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:23.910254 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:23.910760 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 07:39:23.910889 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 07:39:23.911952 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:23.899180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:51081)
0811 07:39:23.899654 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:23.899697 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:23.900984 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:23.901589 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:23.901596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:23.902063 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:23.902723 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:23.902741 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.903711 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.903715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:23.904746 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:23.904757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:23.905285 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:23.905289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:23.905464 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:23.905998 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:23.906014 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:23.907449 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:23.909099 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:23.909105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:23.909121 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:23.909392 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:23.909689 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:23.909693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:23.909695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:23.909849 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:23.910253 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:23.910262 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:23.910519 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 07:39:23.910717 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 07:39:23.911068 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260811 07:39:23.913056 28272 heartbeater.cc:347] Connected to a master server at 127.18.72.254:36435
I20260811 07:39:23.913319 28272 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:23.913857 28272 heartbeater.cc:511] Master 127.18.72.254:36435 requested a full tablet report, sending...
I20260811 07:39:23.915006 27773 ts_manager.cc:194] Registered new tserver with Master: 496de97553804ff8be25d3395c4d12e8 (127.18.72.195:45707)
I20260811 07:39:23.915807 27773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:51081
I20260811 07:39:23.916318 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab.
Entry for principal oryx/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab.
WARNING: no policy specified for oryx/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab.
Entry for principal oryx/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab.
WARNING: no policy specified for oryx/127.18.72.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab.
Entry for principal oryx/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab.
I20260811 07:39:23.998342 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 27841
I20260811 07:39:24.006740 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 27984
I20260811 07:39:24.016304 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 28130
I20260811 07:39:24.024561 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 27739
I20260811 07:39:24.032591 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:36435
--webserver_interface=127.18.72.254
--webserver_port=36219
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:36435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:24.149415 28282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:24.149740 28282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:24.149832 28282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:24.153934 28282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:24.154023 28282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:24.154064 28282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:24.154124 28282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:24.154165 28282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:24.159409 28282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:36435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:36435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=36219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:24.160570 28282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:24.161641 28282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:24.170372 28288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:24.170348 28282 server_base.cc:1034] running on GCE node
W20260811 07:39:24.170320 28290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:24.170320 28287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:24.171305 28282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:24.172570 28282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:24.173769 28282 hybrid_clock.cc:648] HybridClock initialized: now 1786433964173750 us; error 32 us; skew 500 ppm
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:24.176743 28282 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:39:24.177749 28282 webserver.cc:460] Webserver started at http://127.18.72.254:36219/ using document root <none> and password file <none>
I20260811 07:39:24.178361 28282 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:24.178483 28282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:24.182580 28282 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260811 07:39:24.184949 28299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.186281 28282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:24.186450 28282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "30d11aeb9e944ed48e5720e89f5292bb"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "d277c42e854f5266f4746df14a54955a"
server_key_iv: "162f479883ebffb3358890e4e7729538"
server_key_version: "encryptionkey@0"
I20260811 07:39:24.187055 28282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:24.203322 28282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:24.206398 28282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:24.206615 28282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:24.214398 28282 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:36435
I20260811 07:39:24.214437 28354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:36435 every 8 connection(s)
I20260811 07:39:24.215519 28282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:24.219998 28355 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:39:24.220203 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 28282
I20260811 07:39:24.220722 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:43037
--local_ip_for_outbound_sockets=127.18.72.193
--tserver_master_addrs=127.18.72.254:36435
--webserver_port=42659
--webserver_interface=127.18.72.193
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.72.193
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:39:24.222303 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Bootstrap starting.
I20260811 07:39:24.240175 28355 log.cc:824] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:24.247359 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:39:24.247774 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Bootstrap complete.
I20260811 07:39:24.252238 28355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } }
I20260811 07:39:24.252678 28355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d11aeb9e944ed48e5720e89f5292bb, State: Initialized, Role: FOLLOWER
I20260811 07:39:24.253180 28355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } }
I20260811 07:39:24.253290 28355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:24.253382 28355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:24.253525 28355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:39:24.255486 28355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } }
I20260811 07:39:24.255820 28355 leader_election.cc:302] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30d11aeb9e944ed48e5720e89f5292bb; no voters:
I20260811 07:39:24.256170 28355 leader_election.cc:288] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:39:24.256333 28359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:39:24.256564 28359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [term 2 LEADER]: Becoming Leader. State: Replica: 30d11aeb9e944ed48e5720e89f5292bb, State: Running, Role: LEADER
I20260811 07:39:24.256873 28359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } }
I20260811 07:39:24.257436 28355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:24.258337 28360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "30d11aeb9e944ed48e5720e89f5292bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } } }
I20260811 07:39:24.258467 28360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:24.258760 28361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30d11aeb9e944ed48e5720e89f5292bb. Latest consensus state: current_term: 2 leader_uuid: "30d11aeb9e944ed48e5720e89f5292bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d11aeb9e944ed48e5720e89f5292bb" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36435 } } }
I20260811 07:39:24.258841 28361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:24.259526 28368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:24.262737 28368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:24.263401 28368 catalog_manager.cc:1259] Loaded cluster ID: de3ae1dae72542268682e5a37c94229d
I20260811 07:39:24.263463 28368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:24.265910 28368 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:24.266595 28368 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30d11aeb9e944ed48e5720e89f5292bb: Loaded TSK: 0
I20260811 07:39:24.268011 28368 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:39:24.341207 28357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:24.341503 28357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:24.341589 28357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:24.345747 28357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:24.345827 28357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:24.345914 28357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:24.351265 28357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.193.keytab
--principal=oryx/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:43037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=42659
--tserver_master_addrs=127.18.72.254:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:24.352413 28357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:24.353417 28357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:24.363194 28383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:24.363211 28382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:24.363211 28385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:24.364141 28357 server_base.cc:1034] running on GCE node
I20260811 07:39:24.364532 28357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:24.365216 28357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:24.366400 28357 hybrid_clock.cc:648] HybridClock initialized: now 1786433964366370 us; error 43 us; skew 500 ppm
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433964, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:24.369729 28357 init.cc:377] Logged in from keytab as oryx/127.18.72.193@KRBTEST.COM (short username oryx)
I20260811 07:39:24.370754 28357 webserver.cc:460] Webserver started at http://127.18.72.193:42659/ using document root <none> and password file <none>
I20260811 07:39:24.371348 28357 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:24.371461 28357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:24.375733 28357 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:39:24.378139 28392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.379593 28357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:39:24.379746 28357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "d1505d4263bd471c956b17a9e64dc163"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "1b83c5b376c0b1d93f0110aea6723829"
server_key_iv: "3b72835f0d7aa91ec74d2e91914a42ba"
server_key_version: "encryptionkey@0"
I20260811 07:39:24.380416 28357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:24.404212 28357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:24.407766 28357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:24.408052 28357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:24.408735 28357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:24.409638 28357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:24.409686 28357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.409756 28357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:24.409789 28357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.419209 28357 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:43037
I20260811 07:39:24.419234 28509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:43037 every 8 connection(s)
I20260811 07:39:24.420397 28357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:24.429085 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 28357
I20260811 07:39:24.429661 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:46429
--local_ip_for_outbound_sockets=127.18.72.194
--tserver_master_addrs=127.18.72.254:36435
--webserver_port=35485
--webserver_interface=127.18.72.194
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.72.194
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 07:39:24.437075 28512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.423088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:42355 (local address 127.18.72.254:36435)
0811 07:39:24.423652 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.423669 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.424479 (+ 810us) server_negotiation.cc:407] Connection header received
0811 07:39:24.425804 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.425818 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.426197 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.426534 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.428039 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.429479 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.430957 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.431758 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.431971 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.436253 (+ 4282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:42355: BlockingRecv error: recv got EOF from 127.18.72.193:42355 (error 108)
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":170,"threads_started":1}
I20260811 07:39:24.437084 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.423146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:42355)
0811 07:39:24.424245 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.424297 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.425273 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.426703 (+ 1430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.426712 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.427164 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.427738 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.427764 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.429683 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.429687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.430802 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.430816 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.431993 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.432000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.432253 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.433013 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.433040 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.436003 (+ 2963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":173,"threads_started":1}
W20260811 07:39:24.437593 28510 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:36435 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.72.254:36435: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:24.443563 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.438157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:53153)
0811 07:39:24.438289 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.438306 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.438401 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.438681 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.438684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.438862 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.438960 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.438966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.439721 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.439726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.440378 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.440388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.440868 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.440872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.440967 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.441546 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.441559 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.443399 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 07:39:24.443657 28512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.438234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:53153 (local address 127.18.72.254:36435)
0811 07:39:24.438357 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.438362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.438376 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:24.438515 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.438518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.438571 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.438663 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.439079 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.439606 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.440516 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.440740 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.440860 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.443536 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:53153: BlockingRecv error: recv got EOF from 127.18.72.193:53153 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:24.450225 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.444368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:49197)
0811 07:39:24.444544 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.444557 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.444646 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.445086 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.445089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.445262 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.445358 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.445364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.446175 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.446178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.447051 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.447061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.447509 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.447513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.447600 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.448140 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.448160 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.450092 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260811 07:39:24.450358 28512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.444482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:49197 (local address 127.18.72.254:36435)
0811 07:39:24.444594 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.444597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.444609 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:24.444916 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.444919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.444967 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.445041 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.445474 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.446067 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.447172 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.447397 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.447478 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.450218 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:49197: BlockingRecv error: recv got EOF from 127.18.72.193:49197 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 07:39:24.450624 28510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:39:24.550806 28515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:24.551083 28515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:24.551129 28515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:24.555286 28515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:24.555358 28515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:24.555462 28515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:24.561223 28515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.194.keytab
--principal=oryx/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:46429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=35485
--tserver_master_addrs=127.18.72.254:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:24.562340 28515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:24.563453 28515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:24.573624 28520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:24.573634 28523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:24.573907 28521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:24.573772 28515 server_base.cc:1034] running on GCE node
I20260811 07:39:24.574378 28515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:24.575062 28515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:24.576267 28515 hybrid_clock.cc:648] HybridClock initialized: now 1786433964576218 us; error 56 us; skew 500 ppm
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433964, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:24.579468 28515 init.cc:377] Logged in from keytab as oryx/127.18.72.194@KRBTEST.COM (short username oryx)
I20260811 07:39:24.580467 28515 webserver.cc:460] Webserver started at http://127.18.72.194:35485/ using document root <none> and password file <none>
I20260811 07:39:24.581046 28515 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:24.581120 28515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:24.585405 28515 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:39:24.587525 28530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.589023 28515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:24.589125 28515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "7bef2a92ed5143a58d22e465c05b4297"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "24f52610dd082ea190cf6b30af1c791f"
server_key_iv: "8e3e35d29a49f46f3b438746ad6b7396"
server_key_version: "encryptionkey@0"
I20260811 07:39:24.589807 28515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:24.603010 28515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:24.605877 28515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:24.606030 28515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:24.606578 28515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:24.607374 28515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:24.607421 28515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.607461 28515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:24.607476 28515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.617358 28515 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:46429
I20260811 07:39:24.617395 28650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:46429 every 8 connection(s)
I20260811 07:39:24.618487 28515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:24.628475 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 28515
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:24.628978 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:45707
--local_ip_for_outbound_sockets=127.18.72.195
--tserver_master_addrs=127.18.72.254:36435
--webserver_port=39713
--webserver_interface=127.18.72.195
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.72.195
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:24.630152 28512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.620478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:59819 (local address 127.18.72.254:36435)
0811 07:39:24.620628 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.620632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.621770 (+ 1138us) server_negotiation.cc:407] Connection header received
0811 07:39:24.622698 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.622701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.622755 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.622822 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.624138 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.624670 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.625921 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.626127 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.626205 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.629979 (+ 3774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:59819: BlockingRecv error: recv got EOF from 127.18.72.194:59819 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:39:24.630705 28653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.620861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:59819)
0811 07:39:24.621596 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.621643 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.622494 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.622972 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.622979 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.623381 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.623994 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.624012 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.624802 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.624805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.625807 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.625818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.626278 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.626281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.626442 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.627023 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.627042 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.629790 (+ 2748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":115,"threads_started":1}
W20260811 07:39:24.631256 28651 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:36435 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.72.254:36435: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:24.637506 28653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.631753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:48235)
0811 07:39:24.631933 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.631951 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.632035 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.632280 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.632282 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.632421 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.632518 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.632523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.633565 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.633567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.634169 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.634178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.634712 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.634716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.634825 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.635376 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.635389 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.637343 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 07:39:24.637655 28512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.631871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:48235 (local address 127.18.72.254:36435)
0811 07:39:24.632026 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.632032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.632051 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:39:24.632102 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.632105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.632153 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.632240 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.632638 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.633447 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.634328 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.634578 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.634708 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.637480 (+ 2772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:48235: BlockingRecv error: recv got EOF from 127.18.72.194:48235 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:24.644537 28653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.638319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:34159)
0811 07:39:24.638515 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.638533 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.638635 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.638889 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.638902 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.639092 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.639221 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.639230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.640044 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.640049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.641046 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.641062 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.641553 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.641558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.641682 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.642240 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.642255 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.644393 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114}
W20260811 07:39:24.644750 28512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.638404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:34159 (local address 127.18.72.254:36435)
0811 07:39:24.638532 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.638536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.638582 (+ 46us) server_negotiation.cc:407] Connection header received
0811 07:39:24.638729 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.638731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.638770 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.638840 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.639343 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.639877 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.641208 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.641422 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.641512 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.644576 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:34159: BlockingRecv error: recv got EOF from 127.18.72.194:34159 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 07:39:24.645010 28651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:39:24.747236 28654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:24.747496 28654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:24.747539 28654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:24.751474 28654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:24.751526 28654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:24.751617 28654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:24.756739 28654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/krb5kdc/oryx_127.18.72.195.keytab
--principal=oryx/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:45707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=39713
--tserver_master_addrs=127.18.72.254:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:24.757802 28654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:24.758919 28654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:24.768715 28663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:24.768780 28660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:24.768733 28661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:24.769073 28654 server_base.cc:1034] running on GCE node
I20260811 07:39:24.769600 28654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:24.770246 28654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:24.771459 28654 hybrid_clock.cc:648] HybridClock initialized: now 1786433964771424 us; error 45 us; skew 500 ppm
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433964, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:24.774451 28654 init.cc:377] Logged in from keytab as oryx/127.18.72.195@KRBTEST.COM (short username oryx)
I20260811 07:39:24.775332 28654 webserver.cc:460] Webserver started at http://127.18.72.195:39713/ using document root <none> and password file <none>
I20260811 07:39:24.775879 28654 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:24.776033 28654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:24.780076 28654 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 07:39:24.782291 28670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.783547 28654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:24.783644 28654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "496de97553804ff8be25d3395c4d12e8"
format_stamp: "Formatted at 2026-08-11 07:39:23 on dist-test-slave-zkpd"
server_key: "04f1d7c8a108f7f8df3d0a9cb0944233"
server_key_iv: "29923a0336646e522674d3c8dff72f40"
server_key_version: "encryptionkey@0"
I20260811 07:39:24.784296 28654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:24.802904 28654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:24.806283 28654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:24.806515 28654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:24.807240 28654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:24.808200 28654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:24.808271 28654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.808346 28654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:24.808390 28654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:24.817521 28654 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:45707
I20260811 07:39:24.817552 28790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:45707 every 8 connection(s)
I20260811 07:39:24.818555 28654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:24.827417 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 28654
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 07:39:24.830276 28512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.820505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:44159 (local address 127.18.72.254:36435)
0811 07:39:24.820643 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.820646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.821777 (+ 1131us) server_negotiation.cc:407] Connection header received
0811 07:39:24.822710 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.822713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.822773 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.822880 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.824163 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.824701 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.825850 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.826078 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.826198 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.830173 (+ 3975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:44159: BlockingRecv error: recv got EOF from 127.18.72.195:44159 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:39:24.830977 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.820959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:44159)
0811 07:39:24.821629 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.821670 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.822490 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.823069 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.823077 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.823438 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.824015 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.824033 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.824840 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.824844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.825720 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.825730 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.826240 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.826245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.826424 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.826982 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.826998 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.830063 (+ 3065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":116,"threads_started":1}
W20260811 07:39:24.831465 28791 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:36435 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.72.254:36435: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:24.837229 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.831952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:49599)
0811 07:39:24.832066 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.832082 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.832153 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.832369 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.832371 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.832497 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.832639 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.832646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.833675 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.833677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.834252 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.834263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.834829 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.834833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.834943 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.835376 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.835388 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.837095 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 07:39:24.837369 28512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.832016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:49599 (local address 127.18.72.254:36435)
0811 07:39:24.832125 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.832131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.832146 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:39:24.832213 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.832215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.832252 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.832327 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.832783 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.833564 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.834371 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.834707 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.834815 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.837238 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:49599: BlockingRecv error: recv got EOF from 127.18.72.195:49599 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:39:24.838042 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.828517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38136 (local address 127.18.72.254:36435)
0811 07:39:24.828731 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.828734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.828746 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:39:24.828778 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.828780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.828818 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.828898 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.829753 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.830487 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.831634 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.831945 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.832048 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.833891 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:24.833897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:24.833913 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:24.833940 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:24.835690 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:24.836231 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:24.836244 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:24.836249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:24.836373 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:24.836668 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:24.836672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:24.836673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:24.837057 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:24.837267 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:24.837733 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 07:39:24.837933 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:24 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:24.843568 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:24.838471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:50895)
0811 07:39:24.838590 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:24.838605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:24.838672 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:24.839316 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:24.839318 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:24.839440 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:24.839507 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.839511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.840433 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.840439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.840975 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:24.840984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.841440 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.841444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:24.841522 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.841883 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:24.841894 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:24.843441 (+ 1547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 07:39:24.843655 28795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:24.838990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:50895 (local address 127.18.72.254:36435)
0811 07:39:24.839120 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:24.839124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:24.839137 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:24.839167 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:24.839170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:24.839206 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:24.839283 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:24.839611 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.840325 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.841084 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:24.841330 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:24.841437 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:24.843549 (+ 2112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:50895: BlockingRecv error: recv got EOF from 127.18.72.195:50895 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 07:39:24.843845 28791 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 07:39:25 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:25 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:25.457283 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:25.451344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:34733)
0811 07:39:25.451634 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:25.451649 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:25.451742 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:25.452017 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:25.452019 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:25.452207 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:25.452352 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:25.452358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.453116 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.453118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:25.453998 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:25.454011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.454612 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.454615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:25.454687 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:25.455243 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:25.455264 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:25.457169 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":105,"threads_started":1}
W20260811 07:39:25.457533 28798 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:25.451414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:34733 (local address 127.18.72.254:36435)
0811 07:39:25.451683 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:25.451687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:25.451702 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:39:25.451830 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:25.451833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:25.451877 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:25.451976 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:25.452459 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.452983 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.454125 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.454493 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.454586 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:25.457387 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:34733: BlockingRecv error: recv got EOF from 127.18.72.193:34733 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":107,"threads_started":1}
Aug 11 07:39:25 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:25 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:25.651450 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:25.645667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:53227)
0811 07:39:25.645941 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:25.645956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:25.646055 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:25.646311 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:25.646313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:25.646492 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:25.646588 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:25.646593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.647344 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.647346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:25.647945 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:25.647953 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.648348 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.648351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:25.648412 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:25.648973 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:25.648985 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:25.651320 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":75,"threads_started":1}
W20260811 07:39:25.651572 28798 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:25.645749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:53227 (local address 127.18.72.254:36435)
0811 07:39:25.645955 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:25.645959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:25.646033 (+ 74us) server_negotiation.cc:407] Connection header received
0811 07:39:25.646149 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:25.646152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:25.646198 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:25.646275 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:25.646716 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.647244 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.648041 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.648251 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.648334 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:25.651456 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:53227: BlockingRecv error: recv got EOF from 127.18.72.194:53227 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 07:39:25 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:25 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:25.850385 28800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:25.844716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:39405)
0811 07:39:25.845087 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:25.845103 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:25.845224 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:25.845506 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:25.845509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:25.845718 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:25.845863 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:25.845870 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.846626 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.846631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:25.847203 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:25.847211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.847673 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.847676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:25.847762 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:25.848356 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:25.848369 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:25.850206 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":81,"threads_started":1}
W20260811 07:39:25.850464 28798 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:25.844764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:39405 (local address 127.18.72.254:36435)
0811 07:39:25.844892 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:25.844896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:25.845169 (+ 273us) server_negotiation.cc:407] Connection header received
0811 07:39:25.845327 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:25.845338 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:25.845393 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:25.845481 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:25.845978 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.846513 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.847324 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:25.847557 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:25.847667 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:25.850321 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:39405: BlockingRecv error: recv got EOF from 127.18.72.195:39405 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 07:39:26 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:26 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:26.463748 28801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:26.458234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:34833)
0811 07:39:26.458539 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:26.458554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:26.458659 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:26.458946 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:26.458949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:26.459144 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:26.459247 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:26.459253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.460075 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.460078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:26.460702 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:26.460710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.461170 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.461172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:26.461240 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:26.461734 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:26.461747 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:26.463612 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":106,"threads_started":1}
W20260811 07:39:26.463872 28802 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:26.458304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:34833 (local address 127.18.72.254:36435)
0811 07:39:26.458573 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:26.458577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:26.458606 (+ 29us) server_negotiation.cc:407] Connection header received
0811 07:39:26.458741 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:26.458744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:26.458788 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:26.458880 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:26.459420 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.459970 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.460808 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.461057 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.461160 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:26.463746 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:34833: BlockingRecv error: recv got EOF from 127.18.72.193:34833 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":110,"threads_started":1}
Aug 11 07:39:26 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:26 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:26.657930 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:26.652241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:44885)
0811 07:39:26.652592 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:26.652611 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:26.652754 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:26.653081 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:26.653084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:26.653305 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:26.653407 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:26.653413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.654189 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.654194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:26.654754 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:26.654762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.655247 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.655251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:26.655340 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:26.655840 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:26.655852 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:26.657790 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":117,"threads_started":1}
W20260811 07:39:26.658191 28802 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:26.652374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:44885 (local address 127.18.72.254:36435)
0811 07:39:26.652545 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:26.652551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:26.652688 (+ 137us) server_negotiation.cc:407] Connection header received
0811 07:39:26.652908 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:26.652911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:26.652960 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:26.653056 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:26.653532 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.654074 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.654884 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.655125 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.655231 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:26.657944 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:44885: BlockingRecv error: recv got EOF from 127.18.72.194:44885 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 07:39:26 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:26 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:26.862439 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:26.851271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:60811)
0811 07:39:26.851743 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:26.851765 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:26.851925 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:26.852206 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:26.852209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:26.852431 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:26.852525 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:26.852531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.853473 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.853480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:26.854060 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:26.854069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.854569 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.854573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:26.854662 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:26.860390 (+ 5728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:26.860403 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:26.862298 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":130,"threads_started":1}
W20260811 07:39:26.862591 28802 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:26.851456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:60811 (local address 127.18.72.254:36435)
0811 07:39:26.851713 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:26.851719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:26.851840 (+ 121us) server_negotiation.cc:407] Connection header received
0811 07:39:26.852032 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:26.852036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:26.852086 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:26.852184 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:26.852649 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.853332 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.854191 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:26.854442 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:26.854562 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:26.862435 (+ 7873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:60811: BlockingRecv error: recv got EOF from 127.18.72.195:60811 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
Aug 11 07:39:27 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:27 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:27.471489 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:27.464755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:56405)
0811 07:39:27.465116 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:27.465135 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:27.465274 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:27.465613 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:27.465616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:27.465915 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:27.466057 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:27.466066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.467147 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.467151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:27.468071 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:27.468085 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.468795 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.468800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:27.468897 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:27.469496 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:27.469508 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:27.471360 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":103,"threads_started":1}
W20260811 07:39:27.471781 28806 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:27.465071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:56405 (local address 127.18.72.254:36435)
0811 07:39:27.465331 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:27.465335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:27.465348 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:27.465413 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:27.465418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:27.465469 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:27.465570 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:27.466197 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.467015 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.468213 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.468642 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.468772 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:27.471497 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:56405: BlockingRecv error: recv got EOF from 127.18.72.193:56405 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":110,"threads_started":1}
Aug 11 07:39:27 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:27 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:27.664485 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:27.658875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:50557)
0811 07:39:27.659191 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:27.659205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:27.659306 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:27.659528 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:27.659530 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:27.659718 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:27.659873 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:27.659880 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.660650 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.660653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:27.661208 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:27.661215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.661667 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.661669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:27.661731 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:27.662218 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:27.662229 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:27.664370 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":130,"threads_started":1}
W20260811 07:39:27.664664 28806 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:27.658931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:50557 (local address 127.18.72.254:36435)
0811 07:39:27.659110 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:27.659114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:27.659262 (+ 148us) server_negotiation.cc:407] Connection header received
0811 07:39:27.659377 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:27.659379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:27.659423 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:27.659491 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:27.660032 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.660559 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.661343 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.661563 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.661649 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:27.664503 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:50557: BlockingRecv error: recv got EOF from 127.18.72.194:50557 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 07:39:27 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:27 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:27.868368 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:27.863251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:54051)
0811 07:39:27.863510 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:27.863523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:27.863637 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:27.863910 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:27.863913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:27.864095 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:27.864194 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:27.864199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.864915 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.864917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:27.865462 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:27.865469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.865859 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.865867 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:27.865930 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:27.866429 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:27.866442 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:27.868245 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":71,"threads_started":1}
W20260811 07:39:27.868479 28806 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:27.863334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:54051 (local address 127.18.72.254:36435)
0811 07:39:27.863512 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:27.863516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:27.863574 (+ 58us) server_negotiation.cc:407] Connection header received
0811 07:39:27.863724 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:27.863726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:27.863782 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:27.863846 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:27.864291 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.864821 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.865560 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:27.865764 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:27.865836 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:27.868367 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:54051: BlockingRecv error: recv got EOF from 127.18.72.195:54051 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 07:39:28 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:28 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:28.478183 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:28.472405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:54757)
0811 07:39:28.472845 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:28.472863 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:28.472993 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:28.473282 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:28.473285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:28.473505 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:28.473602 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:28.473608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.474448 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.474453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:28.475019 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:28.475028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.475518 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.475521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:28.475611 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:28.476239 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:28.476251 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:28.478068 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":107,"threads_started":1}
W20260811 07:39:28.478420 28812 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:28.472441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:54757 (local address 127.18.72.254:36435)
0811 07:39:28.472741 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:28.472747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:28.472952 (+ 205us) server_negotiation.cc:407] Connection header received
0811 07:39:28.473104 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:28.473106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:28.473153 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:28.473254 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:28.473778 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.474330 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.475145 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.475389 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.475500 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:28.478267 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:54757: BlockingRecv error: recv got EOF from 127.18.72.193:54757 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":105,"threads_started":1}
Aug 11 07:39:28 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:28 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:28.671401 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:28.665307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:37563)
0811 07:39:28.665591 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:28.665605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:28.665743 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:28.666026 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:28.666029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:28.666217 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:28.666318 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:28.666324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.667121 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.667125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:28.667768 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:28.667778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.668277 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.668280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:28.668350 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:28.668965 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:28.668980 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:28.671236 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":76,"threads_started":1}
W20260811 07:39:28.671505 28812 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:28.665408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:37563 (local address 127.18.72.254:36435)
0811 07:39:28.665608 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:28.665612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:28.665678 (+ 66us) server_negotiation.cc:407] Connection header received
0811 07:39:28.665833 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:28.665835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:28.665881 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:28.665952 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:28.666457 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.666989 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.667918 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.668153 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.668236 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:28.671394 (+ 3158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:37563: BlockingRecv error: recv got EOF from 127.18.72.194:37563 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 07:39:28 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:28 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:28.874588 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:28.869181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:38347)
0811 07:39:28.869454 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:28.869468 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:28.869555 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:28.869821 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:28.869823 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:28.870005 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:28.870105 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:28.870110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.870886 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.870889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:28.871437 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:28.871445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.871956 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.871960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:28.872034 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:28.872610 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:28.872622 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:28.874475 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":98,"threads_started":1}
W20260811 07:39:28.874756 28812 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:28.869335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:38347 (local address 127.18.72.254:36435)
0811 07:39:28.869493 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:28.869497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:28.869512 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:39:28.869648 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:28.869650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:28.869703 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:28.869786 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:28.870224 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.870787 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.871545 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:28.871799 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:28.871913 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:28.874602 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:38347: BlockingRecv error: recv got EOF from 127.18.72.195:38347 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 07:39:29 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:29 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:29.484349 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:29.478954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:36359)
0811 07:39:29.479236 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:29.479248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:29.479346 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:29.479582 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:29.479585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:29.479769 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:29.479871 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:29.479876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.480649 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.480652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:29.481234 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:29.481245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.481668 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.481670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:29.481734 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:29.482201 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:29.482212 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:29.484171 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":68,"threads_started":1}
W20260811 07:39:29.484462 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:29.479026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:36359 (local address 127.18.72.254:36435)
0811 07:39:29.479294 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:29.479298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:29.479311 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:29.479413 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:29.479415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:29.479455 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:29.479530 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:29.480019 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.480551 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.481350 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.481571 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.481645 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:29.484294 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:36359: BlockingRecv error: recv got EOF from 127.18.72.193:36359 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":116,"threads_started":1}
Aug 11 07:39:29 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:29 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:29.677950 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:29.672360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:55613)
0811 07:39:29.672761 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:29.672773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:29.672899 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:29.673181 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:29.673184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:29.673419 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:29.673518 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:29.673524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.674300 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.674305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:29.674879 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:29.674888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.675360 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.675364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:29.675454 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:29.676000 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:29.676012 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:29.677833 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":89,"threads_started":1}
W20260811 07:39:29.678093 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:29.672475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:55613 (local address 127.18.72.254:36435)
0811 07:39:29.672616 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:29.672622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:29.672869 (+ 247us) server_negotiation.cc:407] Connection header received
0811 07:39:29.672992 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:29.672995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:29.673047 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:29.673153 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:29.673639 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.674185 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.675003 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.675235 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.675347 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:29.677971 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:55613: BlockingRecv error: recv got EOF from 127.18.72.194:55613 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 11 07:39:29 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:29 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:29.881636 28819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:29.875530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:51211)
0811 07:39:29.875822 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:29.875837 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:29.875967 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:29.876223 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:29.876225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:29.876443 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:29.876546 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:29.876551 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.877254 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.877256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:29.878079 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:29.878091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.878691 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.878698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:29.878796 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:29.879325 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:29.879338 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:29.881523 (+ 2185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":95,"threads_started":1}
W20260811 07:39:29.881811 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:29.875584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:51211 (local address 127.18.72.254:36435)
0811 07:39:29.875729 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:29.875733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:29.875908 (+ 175us) server_negotiation.cc:407] Connection header received
0811 07:39:29.876048 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:29.876050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:29.876096 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:29.876180 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:29.876648 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.877158 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.878204 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:29.878544 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:29.878667 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:29.881664 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:51211: BlockingRecv error: recv got EOF from 127.18.72.195:51211 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 07:39:30 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:30 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:30.491024 28820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:30.485074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:47121)
0811 07:39:30.485404 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:30.485423 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:30.485542 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:30.485794 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:30.485796 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:30.486016 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:30.486125 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:30.486131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.486859 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.486862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:30.487507 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:30.487516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.488061 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.488064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:30.488137 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:30.488745 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:30.488765 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:30.490907 (+ 2142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":67,"threads_started":1}
W20260811 07:39:30.491180 28821 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:30.485321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:47121 (local address 127.18.72.254:36435)
0811 07:39:30.485546 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:30.485549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:30.485562 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:30.485617 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:30.485619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:30.485658 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:30.485731 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:30.486225 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.486747 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.487665 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.487917 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.487995 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:30.491040 (+ 3045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:47121: BlockingRecv error: recv got EOF from 127.18.72.193:47121 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":80,"threads_started":1}
Aug 11 07:39:30 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:30 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:30.684595 28822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:30.678821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:39153)
0811 07:39:30.679191 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:30.679205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:30.679363 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:30.679649 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:30.679652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:30.679871 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:30.680001 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:30.680007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.680775 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.680779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:30.681346 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:30.681355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.681834 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.681838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:30.681930 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:30.682571 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:30.682582 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:30.684481 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":86,"threads_started":1}
W20260811 07:39:30.684734 28821 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:30.678954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:39153 (local address 127.18.72.254:36435)
0811 07:39:30.679143 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:30.679147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:30.679284 (+ 137us) server_negotiation.cc:407] Connection header received
0811 07:39:30.679468 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:30.679472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:30.679527 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:30.679621 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:30.680126 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.680658 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.681470 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.681700 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.681827 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:30.684616 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:39153: BlockingRecv error: recv got EOF from 127.18.72.194:39153 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 07:39:30 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:30 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:30.888015 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:30.882692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:34919)
0811 07:39:30.882956 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:30.882969 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:30.883070 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:30.883296 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:30.883299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:30.883485 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:30.883579 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:30.883584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.884395 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.884397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:30.884992 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:30.885001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.885493 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.885495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:30.885559 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:30.886026 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:30.886038 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:30.887817 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":85,"threads_started":1}
W20260811 07:39:30.888165 28821 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:30.882739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:34919 (local address 127.18.72.254:36435)
0811 07:39:30.882864 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:30.882868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:30.883033 (+ 165us) server_negotiation.cc:407] Connection header received
0811 07:39:30.883146 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:30.883148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:30.883193 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:30.883260 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:30.883671 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.884293 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.885097 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:30.885389 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:30.885470 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:30.888047 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:34919: BlockingRecv error: recv got EOF from 127.18.72.195:34919 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 07:39:31 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:31 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:31.497635 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:31.491821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:38383)
0811 07:39:31.492173 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:31.492188 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:31.492287 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:31.492584 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:31.492586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:31.492766 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:31.492865 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:31.492870 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.493704 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.493707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:31.494359 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:31.494368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.494833 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.494837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:31.494915 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:31.495408 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:31.495420 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:31.497462 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":106,"threads_started":1}
W20260811 07:39:31.497709 28825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:31.491872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:38383 (local address 127.18.72.254:36435)
0811 07:39:31.492170 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:31.492173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:31.492256 (+ 83us) server_negotiation.cc:407] Connection header received
0811 07:39:31.492414 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:31.492417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:31.492466 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:31.492552 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:31.492963 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.493588 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.494478 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.494716 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.494828 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:31.497584 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:38383: BlockingRecv error: recv got EOF from 127.18.72.193:38383 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":81,"threads_started":1}
Aug 11 07:39:31 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:31 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:31.690886 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:31.685530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:41699)
0811 07:39:31.685874 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:31.685892 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:31.686002 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:31.686230 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:31.686232 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:31.686426 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:31.686523 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:31.686529 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.687252 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.687255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:31.687810 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:31.687817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.688281 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.688287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:31.688370 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:31.688941 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:31.688952 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:31.690772 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":106,"threads_started":1}
W20260811 07:39:31.690932 28825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:31.685590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:41699 (local address 127.18.72.254:36435)
0811 07:39:31.685750 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:31.685754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:31.685969 (+ 215us) server_negotiation.cc:407] Connection header received
0811 07:39:31.686077 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:31.686079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:31.686122 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:31.686191 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:31.686619 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.687151 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.687935 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.688149 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.688252 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:31.690841 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:41699: BlockingRecv error: recv got EOF from 127.18.72.194:41699 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 07:39:31 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:31 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:31.898660 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:31.888793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:56039)
0811 07:39:31.889192 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:31.889220 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:31.889359 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:31.889653 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:31.889656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:31.889883 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:31.889977 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:31.889982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.890795 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.890800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:31.891366 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:31.891374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.891856 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.891860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:31.891980 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:31.896400 (+ 4420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:31.896414 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:31.898460 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":105,"threads_started":1}
W20260811 07:39:31.898794 28825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:31.888839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:56039 (local address 127.18.72.254:36435)
0811 07:39:31.889065 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:31.889070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:31.889299 (+ 229us) server_negotiation.cc:407] Connection header received
0811 07:39:31.889478 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:31.889481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:31.889539 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:31.889624 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:31.890096 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.890650 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.891492 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:31.891739 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:31.891851 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:31.898648 (+ 6797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:56039: BlockingRecv error: recv got EOF from 127.18.72.195:56039 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 07:39:32 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:32 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:32.504004 28828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:32.498698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:60781)
0811 07:39:32.498995 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:32.499012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:32.499124 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:32.499404 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:32.499407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:32.499615 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:32.499717 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:32.499724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.500516 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.500519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:32.501094 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:32.501102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.501519 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.501522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:32.501602 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:32.502052 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:32.502063 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:32.503827 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":77,"threads_started":1}
W20260811 07:39:32.504133 28829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:32.498806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:60781 (local address 127.18.72.254:36435)
0811 07:39:32.499088 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:32.499094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:32.499109 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:39:32.499224 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:32.499227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:32.499278 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:32.499378 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:32.499825 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.500417 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.501196 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.501424 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.501496 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:32.504006 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:60781: BlockingRecv error: recv got EOF from 127.18.72.193:60781 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":73,"threads_started":1}
Aug 11 07:39:32 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:32 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:32.697271 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:32.691714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:46313)
0811 07:39:32.692003 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:32.692019 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:32.692127 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:32.692346 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:32.692348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:32.692535 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:32.692634 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:32.692639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.693603 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.693606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:32.694226 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:32.694243 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.694711 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.694713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:32.694775 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:32.695336 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:32.695349 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:32.697160 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":82,"threads_started":1}
W20260811 07:39:32.697420 28829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:32.691773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:46313 (local address 127.18.72.254:36435)
0811 07:39:32.691967 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:32.691971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:32.692076 (+ 105us) server_negotiation.cc:407] Connection header received
0811 07:39:32.692193 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:32.692195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:32.692237 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:32.692309 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:32.692771 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.693502 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.694343 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.694612 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.694694 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:32.697294 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:46313: BlockingRecv error: recv got EOF from 127.18.72.194:46313 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 07:39:32 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:32 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:32.904714 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:32.899556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:41065)
0811 07:39:32.899851 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:32.899864 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:32.900033 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:32.900225 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:32.900227 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:32.900405 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:32.900499 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:32.900503 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.901213 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.901215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:32.901746 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:32.901753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.902176 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.902179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:32.902239 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:32.902720 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:32.902731 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:32.904607 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":114,"threads_started":1}
W20260811 07:39:32.904855 28829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:32.899605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:41065 (local address 127.18.72.254:36435)
0811 07:39:32.899764 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:32.899768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:32.899972 (+ 204us) server_negotiation.cc:407] Connection header received
0811 07:39:32.900102 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:32.900104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:32.900150 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:32.900214 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:32.900588 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.901109 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.901856 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:32.902081 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:32.902157 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:32.904734 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:41065: BlockingRecv error: recv got EOF from 127.18.72.195:41065 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 07:39:33 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:33 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:33.510620 28832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:33.504796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:54959)
0811 07:39:33.505060 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:33.505085 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:33.505171 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:33.505415 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:33.505417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:33.505611 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:33.505709 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:33.505713 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.506498 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.506502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:33.507177 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:33.507202 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.507690 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.507694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:33.507782 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:33.508405 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:33.508427 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:33.510450 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
W20260811 07:39:33.510712 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:33.504909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:54959 (local address 127.18.72.254:36435)
0811 07:39:33.505150 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:33.505153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:33.505166 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:33.505249 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:33.505251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:33.505293 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:33.505381 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:33.505808 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.506345 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.507326 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.507566 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.507681 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:33.510581 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:54959: BlockingRecv error: recv got EOF from 127.18.72.193:54959 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":86,"threads_started":1}
Aug 11 07:39:33 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:33 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:33.703234 28834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:33.697991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:47007)
0811 07:39:33.698272 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:33.698298 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:33.698398 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:33.698611 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:33.698614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:33.698790 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:33.698892 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:33.698897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.699590 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.699592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:33.700227 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:33.700236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.700690 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.700694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:33.700762 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:33.701322 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:33.701335 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:33.703105 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":116,"threads_started":1}
W20260811 07:39:33.703315 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:33.698100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:47007 (local address 127.18.72.254:36435)
0811 07:39:33.698238 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:33.698242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:33.698367 (+ 125us) server_negotiation.cc:407] Connection header received
0811 07:39:33.698466 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:33.698468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:33.698510 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:33.698576 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:33.698991 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.699500 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.700336 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.700563 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.700643 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:33.703228 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:47007: BlockingRecv error: recv got EOF from 127.18.72.194:47007 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 07:39:33 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:33 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:33.911532 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:33.905565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:50545)
0811 07:39:33.905876 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:33.905890 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:33.905993 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:33.906266 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:33.906271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:33.906516 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:33.906666 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:33.906674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.907634 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.907637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:33.908354 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:33.908364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.908844 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.908847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:33.908906 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:33.909427 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:33.909438 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:33.911401 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":114,"threads_started":1}
W20260811 07:39:33.911690 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:33.905632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:50545 (local address 127.18.72.254:36435)
0811 07:39:33.905821 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:33.905826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:33.905962 (+ 136us) server_negotiation.cc:407] Connection header received
0811 07:39:33.906068 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:33.906071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:33.906130 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:33.906218 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:33.906798 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.907533 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.908467 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:33.908684 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:33.908830 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:33.911536 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:50545: BlockingRecv error: recv got EOF from 127.18.72.195:50545 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 07:39:34 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:34 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:34.516960 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:34.511494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:48797)
0811 07:39:34.511779 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:34.511791 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:34.511973 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:34.512301 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:34.512307 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:34.512498 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:34.512597 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:34.512602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.513323 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.513325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:34.513902 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:34.513909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.514331 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.514335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:34.514396 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:34.514955 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:34.514966 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:34.516822 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":67,"threads_started":1}
W20260811 07:39:34.517088 28837 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:34.511549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:48797 (local address 127.18.72.254:36435)
0811 07:39:34.511804 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:34.511811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:34.511908 (+ 97us) server_negotiation.cc:407] Connection header received
0811 07:39:34.512099 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:34.512102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:34.512158 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:34.512251 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:34.512696 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.513232 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.514006 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.514224 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.514313 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:34.516957 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:48797: BlockingRecv error: recv got EOF from 127.18.72.193:48797 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
Aug 11 07:39:34 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:34 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:34.709812 28838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:34.704188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:43885)
0811 07:39:34.704468 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:34.704481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:34.704573 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:34.704826 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:34.704828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:34.705007 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:34.705102 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:34.705107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.706102 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.706107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:34.706855 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:34.706867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.707267 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.707270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:34.707332 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:34.707816 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:34.707827 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:34.709698 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":91,"threads_started":1}
W20260811 07:39:34.709944 28837 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:34.704266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:43885 (local address 127.18.72.254:36435)
0811 07:39:34.704481 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:34.704485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:34.704555 (+ 70us) server_negotiation.cc:407] Connection header received
0811 07:39:34.704655 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:34.704657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:34.704702 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:34.704770 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:34.705200 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.705960 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.706966 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.707173 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.707251 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:34.709826 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:43885: BlockingRecv error: recv got EOF from 127.18.72.194:43885 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 11 07:39:34 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:34 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:34.917469 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:34.912398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:47991)
0811 07:39:34.912667 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:34.912680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:34.912768 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:34.913004 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:34.913006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:34.913184 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:34.913280 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:34.913285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.914007 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.914009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:34.914540 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:34.914547 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.914955 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.914958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:34.915018 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:34.915471 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:34.915482 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:34.917358 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":103,"threads_started":1}
W20260811 07:39:34.917634 28837 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:34.912541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:47991 (local address 127.18.72.254:36435)
0811 07:39:34.912674 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:34.912677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:34.912739 (+ 62us) server_negotiation.cc:407] Connection header received
0811 07:39:34.912838 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:34.912840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:34.912882 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:34.912947 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:34.913376 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.913913 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.914632 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:34.914850 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:34.914936 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:34.917506 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:47991: BlockingRecv error: recv got EOF from 127.18.72.195:47991 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 07:39:35 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:35 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:35.523573 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:35.517829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:45517)
0811 07:39:35.518110 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:35.518124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:35.518227 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:35.518478 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:35.518480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:35.518683 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:35.518852 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:35.518864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.519930 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.519936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:35.520497 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:35.520505 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.520988 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.520992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:35.521077 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:35.521532 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:35.521544 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:35.523457 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":73,"threads_started":1}
W20260811 07:39:35.523766 28841 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:35.517856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:45517 (local address 127.18.72.254:36435)
0811 07:39:35.518141 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:35.518145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:35.518176 (+ 31us) server_negotiation.cc:407] Connection header received
0811 07:39:35.518317 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:35.518319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:35.518364 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:35.518444 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:35.518995 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.519809 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.520627 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.520871 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.520982 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:35.523594 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:45517: BlockingRecv error: recv got EOF from 127.18.72.193:45517 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":102,"threads_started":1}
Aug 11 07:39:35 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:35 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:35.716641 28842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:35.710715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:36969)
0811 07:39:35.711027 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:35.711040 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:35.711181 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:35.711470 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:35.711473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:35.711693 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:35.711793 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:35.711799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.712582 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.712588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:35.713149 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:35.713157 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.713627 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.713631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:35.713735 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:35.714408 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:35.714425 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:35.716526 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":125,"threads_started":1}
W20260811 07:39:35.716820 28841 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:35.710790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:36969 (local address 127.18.72.254:36435)
0811 07:39:35.710967 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:35.710972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:35.711119 (+ 147us) server_negotiation.cc:407] Connection header received
0811 07:39:35.711288 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:35.711291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:35.711349 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:35.711441 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:35.711934 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.712461 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.713274 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.713504 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.713611 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:35.716664 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:36969: BlockingRecv error: recv got EOF from 127.18.72.194:36969 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 07:39:35 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:35 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:35.924041 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:35.918291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:57645)
0811 07:39:35.918548 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:35.918564 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:35.918675 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:35.918947 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:35.918950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:35.919190 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:35.919327 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:35.919335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.920398 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.920401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:35.920954 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:35.920963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.921356 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.921358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:35.921421 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:35.921923 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:35.921936 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:35.923869 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":94,"threads_started":1}
W20260811 07:39:35.924176 28841 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:35.918405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:57645 (local address 127.18.72.254:36435)
0811 07:39:35.918538 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:35.918542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:35.918640 (+ 98us) server_negotiation.cc:407] Connection header received
0811 07:39:35.918754 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:35.918756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:35.918801 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:35.918890 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:35.919455 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.920280 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.921051 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:35.921261 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:35.921336 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:35.924056 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:57645: BlockingRecv error: recv got EOF from 127.18.72.195:57645 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 07:39:36 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:36 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:36.530249 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:36.524393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:51601)
0811 07:39:36.524729 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:36.524748 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:36.524891 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:36.525220 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:36.525223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:36.525476 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:36.525614 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:36.525622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.526525 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.526527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:36.527082 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:36.527089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.527510 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.527512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:36.527575 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:36.528199 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:36.528213 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:36.530079 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":111,"threads_started":1}
W20260811 07:39:36.530444 28845 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:36.524525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:51601 (local address 127.18.72.254:36435)
0811 07:39:36.524863 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:36.524867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:36.524881 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:39:36.525014 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:36.525017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:36.525074 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:36.525175 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:36.525753 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.526428 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.527176 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.527412 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.527487 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:36.530198 (+ 2711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:51601: BlockingRecv error: recv got EOF from 127.18.72.193:51601 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":107,"threads_started":1}
Aug 11 07:39:36 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:36 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:36.723147 28846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:36.717469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:32821)
0811 07:39:36.717767 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:36.717781 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:36.717884 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:36.718139 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:36.718141 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:36.718343 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:36.718446 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:36.718452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.719227 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.719230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:36.719780 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:36.719788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.720307 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.720311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:36.720385 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:36.720988 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:36.721001 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:36.722981 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":112,"threads_started":1}
W20260811 07:39:36.723289 28845 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:36.717537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:32821 (local address 127.18.72.254:36435)
0811 07:39:36.717676 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:36.717680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:36.717872 (+ 192us) server_negotiation.cc:407] Connection header received
0811 07:39:36.717970 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:36.717973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:36.718020 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:36.718101 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:36.718571 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.719132 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.719885 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.720169 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.720260 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:36.723174 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:32821: BlockingRecv error: recv got EOF from 127.18.72.194:32821 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 07:39:36 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:36 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:36.934352 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:36.924859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:58527)
0811 07:39:36.925144 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:36.925158 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:36.925261 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:36.925567 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:36.925573 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:36.925817 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:36.925922 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:36.925928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.926647 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.926649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:36.927205 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:36.927213 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.927615 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.927618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:36.927678 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:36.932095 (+ 4417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:36.932113 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:36.934225 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":98,"threads_started":1}
W20260811 07:39:36.934552 28845 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:36.924916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:58527 (local address 127.18.72.254:36435)
0811 07:39:36.925064 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:36.925069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:36.925227 (+ 158us) server_negotiation.cc:407] Connection header received
0811 07:39:36.925356 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:36.925358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:36.925404 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:36.925510 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:36.926026 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.926548 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.927306 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:36.927519 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:36.927594 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:36.934385 (+ 6791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:58527: BlockingRecv error: recv got EOF from 127.18.72.195:58527 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 07:39:37 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:37 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:37.536188 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:37.531020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:38117)
0811 07:39:37.531315 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:37.531329 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:37.531421 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:37.531685 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:37.531687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:37.531899 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:37.532024 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:37.532030 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.532771 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.532773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:37.533335 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:37.533342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.533758 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.533760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:37.533823 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:37.534291 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:37.534302 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:37.536073 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
W20260811 07:39:37.536335 28850 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:37.531112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:38117 (local address 127.18.72.254:36435)
0811 07:39:37.531359 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:37.531363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:37.531376 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:37.531507 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:37.531510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:37.531555 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:37.531636 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:37.532128 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.532677 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.533431 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.533648 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.533739 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:37.536204 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:38117: BlockingRecv error: recv got EOF from 127.18.72.193:38117 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":84,"threads_started":1}
Aug 11 07:39:37 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:37 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:37.730331 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:37.724021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:39773)
0811 07:39:37.724372 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:37.724398 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:37.724548 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:37.724891 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:37.724894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:37.725144 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:37.725262 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:37.725269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.726179 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.726184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:37.726900 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:37.726916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.727462 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.727467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:37.727567 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:37.728112 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:37.728125 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:37.730140 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":87,"threads_started":1}
W20260811 07:39:37.730397 28850 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:37.724153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:39773 (local address 127.18.72.254:36435)
0811 07:39:37.724343 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:37.724348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:37.724481 (+ 133us) server_negotiation.cc:407] Connection header received
0811 07:39:37.724659 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:37.724663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:37.724753 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:37.724862 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:37.725393 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.726047 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.727050 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.727333 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.727457 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:37.730277 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:39773: BlockingRecv error: recv got EOF from 127.18.72.194:39773 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 07:39:37 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:37 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:37.940832 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:37.935130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:58915)
0811 07:39:37.935406 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:37.935419 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:37.935519 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:37.935761 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:37.935764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:37.936029 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:37.936131 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:37.936137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.936879 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.936881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:37.937451 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:37.937458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.937886 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.937889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:37.937959 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:37.938579 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:37.938591 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:37.940663 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
W20260811 07:39:37.940987 28850 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:37.935261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:58915 (local address 127.18.72.254:36435)
0811 07:39:37.935407 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:37.935411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:37.935491 (+ 80us) server_negotiation.cc:407] Connection header received
0811 07:39:37.935601 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:37.935603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:37.935651 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:37.935729 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:37.936237 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.936770 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.937549 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:37.937758 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:37.937852 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:37.940829 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:58915: BlockingRecv error: recv got EOF from 127.18.72.195:58915 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 07:39:38 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:38 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:38.542511 28853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:38.537068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:60235)
0811 07:39:38.537376 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:38.537390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:38.537481 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:38.537746 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:38.537749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:38.537952 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:38.538051 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:38.538056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.538847 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.538850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:38.539407 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:38.539415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.539847 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.539850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:38.539946 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:38.540447 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:38.540459 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:38.542394 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":127,"threads_started":1}
W20260811 07:39:38.542665 28854 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:38.537165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:60235 (local address 127.18.72.254:36435)
0811 07:39:38.537463 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:38.537467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:38.537480 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:39:38.537567 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:38.537569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:38.537619 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:38.537696 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:38.538168 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.538713 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.539517 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.539741 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.539820 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:38.542532 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:60235: BlockingRecv error: recv got EOF from 127.18.72.193:60235 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":91,"threads_started":1}
Aug 11 07:39:38 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:38 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:38.737799 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:38.731245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:51835)
0811 07:39:38.731658 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:38.731672 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:38.731820 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:38.732141 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:38.732145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:38.732380 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:38.732483 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:38.732489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.733334 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.733339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:38.734040 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:38.734050 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.734572 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.734576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:38.734669 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:38.735292 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:38.735306 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:38.737560 (+ 2254us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":119,"threads_started":1}
W20260811 07:39:38.737812 28854 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:38.731325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:51835 (local address 127.18.72.254:36435)
0811 07:39:38.731492 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:38.731497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:38.731760 (+ 263us) server_negotiation.cc:407] Connection header received
0811 07:39:38.731959 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:38.731962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:38.732021 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:38.732117 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:38.732612 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.733208 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.734178 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.734441 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.734564 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:38.737700 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:51835: BlockingRecv error: recv got EOF from 127.18.72.194:51835 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 07:39:38 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:38 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.195@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:38.946952 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:38.941635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.195:52803)
0811 07:39:38.941901 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:38.941914 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:38.942018 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:38.942238 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:38.942240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:38.942433 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:38.942552 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:38.942558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.943260 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.943262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:38.943853 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:38.943861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.944373 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.944377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:38.944470 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:38.944964 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:38.944975 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:38.946763 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":87,"threads_started":1}
W20260811 07:39:38.946992 28854 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:38.941698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:52803 (local address 127.18.72.254:36435)
0811 07:39:38.941830 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:38.941834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:38.941991 (+ 157us) server_negotiation.cc:407] Connection header received
0811 07:39:38.942085 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:38.942087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:38.942133 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:38.942200 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:38.942649 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.943165 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.944008 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:38.944236 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:38.944363 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:38.946885 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.195:52803: BlockingRecv error: recv got EOF from 127.18.72.195:52803 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 07:39:39 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:39 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.193@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:39.549270 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:39.543550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.193:34155)
0811 07:39:39.543831 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:39.543849 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:39.544007 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:39.544264 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:39.544267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:39.544476 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:39.544624 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:39.544632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:39.545389 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:39.545392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:39.546069 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:39.546078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:39.546503 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:39.546506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:39.546575 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:39.547095 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:39.547107 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:39.549145 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":82,"threads_started":1}
W20260811 07:39:39.549454 28858 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:39.543639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:34155 (local address 127.18.72.254:36435)
0811 07:39:39.543947 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:39.543952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:39.543967 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:39:39.544101 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:39.544103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:39.544146 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:39.544224 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:39.544735 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:39.545284 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:39.546177 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:39.546401 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:39.546477 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:39.549274 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.193:34155: BlockingRecv error: recv got EOF from 127.18.72.193:34155 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":63,"threads_started":1}
Aug 11 07:39:39 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 07:39:39 dist-test-slave-zkpd krb5kdc[27726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.72.194@KRBTEST.COM for oryx/127.18.72.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 07:39:39.744783 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:39.738983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36435 (local address 127.18.72.194:33457)
0811 07:39:39.739299 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:39.739312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:39.739453 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:39.739742 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:39.739746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:39.740034 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:39.740133 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:39.740140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:39.740906 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:39.740912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:39.741526 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:39.741535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:39.742003 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:39.742007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:39.742096 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:39.742610 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:39.742621 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:39.744604 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.72.254:36435: Server oryx/127.18.72.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
W20260811 07:39:39.744824 28858 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:39.739064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:33457 (local address 127.18.72.254:36435)
0811 07:39:39.739250 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:39.739255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:39.739393 (+ 138us) server_negotiation.cc:407] Connection header received
0811 07:39:39.739552 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:39.739556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:39.739609 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:39.739714 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:39.740254 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:39.740784 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:39.741659 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:39.741889 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:39.741996 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:39.744725 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.72.194:33457: BlockingRecv error: recv got EOF from 127.18.72.194:33457 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 07:39:39.837551 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 28357
I20260811 07:39:39.848664 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 28515
I20260811 07:39:39.856343 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 28654
I20260811 07:39:39.864718 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 28282
2026-08-11T07:39:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18144 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T07:39:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:39Z Disabled control of system clock
I20260811 07:39:39.899046 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:36127
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:36127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 07:39:40.013216 28863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:40.013470 28863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:40.013502 28863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:40.017443 28863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:40.017498 28863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:40.017513 28863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:40.017529 28863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:40.017544 28863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:40.022123 28863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:36127
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:36127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:40.023072 28863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:40.024776 28863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:40.034039 28869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.034047 28870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.034202 28872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:40.034240 28863 server_base.cc:1034] running on GCE node
I20260811 07:39:40.034909 28863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:40.036070 28863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:40.037259 28863 hybrid_clock.cc:648] HybridClock initialized: now 1786433980037243 us; error 32 us; skew 500 ppm
I20260811 07:39:40.039120 28863 webserver.cc:460] Webserver started at http://127.18.72.254:36061/ using document root <none> and password file <none>
I20260811 07:39:40.039711 28863 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:40.039803 28863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:40.040072 28863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:40.042109 28863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "520a7ca379b34aef84d0343411d79e55"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "5c84e6b5a4c20571c047fd72fafaa17c"
server_key_iv: "30ed01866caa38f410b02ec839952951"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.042685 28863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "520a7ca379b34aef84d0343411d79e55"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "5c84e6b5a4c20571c047fd72fafaa17c"
server_key_iv: "30ed01866caa38f410b02ec839952951"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.046442 28863 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:39:40.049000 28878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.050216 28863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:40.050359 28863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "520a7ca379b34aef84d0343411d79e55"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "5c84e6b5a4c20571c047fd72fafaa17c"
server_key_iv: "30ed01866caa38f410b02ec839952951"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.050470 28863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:40.064999 28863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:40.065659 28863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:40.065855 28863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:40.072965 28863 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:36127
I20260811 07:39:40.072979 28934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:36127 every 8 connection(s)
I20260811 07:39:40.073936 28863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:40.074954 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 28863
I20260811 07:39:40.075099 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:40.075423 18723 external_mini_cluster.cc:1428] Setting key 76aecc9f8ee82f5bea6dd758d0d08b56
I20260811 07:39:40.077440 28935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:40.082969 28935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55: Bootstrap starting.
I20260811 07:39:40.084635 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:40.076903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.18.72.254:36127)
0811 07:39:40.077554 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:40.077599 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:40.077644 (+ 45us) server_negotiation.cc:407] Connection header received
0811 07:39:40.078313 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:40.078330 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:40.078700 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:40.079055 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:40.079470 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.080507 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.081385 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.081851 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.082015 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:40.082909 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:40.082914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:40.082928 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:40.083345 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:40.083399 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:40.083419 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:40.083527 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:40.083643 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 07:39:40.083947 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":151,"threads_started":1}
I20260811 07:39:40.085407 28935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:40.086184 28935 log.cc:824] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:40.088397 28935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55: No bootstrap required, opened a new log
I20260811 07:39:40.092350 28935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520a7ca379b34aef84d0343411d79e55" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36127 } }
I20260811 07:39:40.092597 28935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:40.092681 28935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520a7ca379b34aef84d0343411d79e55, State: Initialized, Role: FOLLOWER
I20260811 07:39:40.093108 28935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520a7ca379b34aef84d0343411d79e55" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36127 } }
I20260811 07:39:40.093261 28935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:40.093338 28935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:40.093443 28935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:40.094450 28935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520a7ca379b34aef84d0343411d79e55" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36127 } }
I20260811 07:39:40.094820 28935 leader_election.cc:302] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 520a7ca379b34aef84d0343411d79e55; no voters:
I20260811 07:39:40.095146 28935 leader_election.cc:288] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:40.095306 28940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:40.095573 28940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [term 1 LEADER]: Becoming Leader. State: Replica: 520a7ca379b34aef84d0343411d79e55, State: Running, Role: LEADER
I20260811 07:39:40.095975 28940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520a7ca379b34aef84d0343411d79e55" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36127 } }
I20260811 07:39:40.096483 28935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:40.097863 28941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "520a7ca379b34aef84d0343411d79e55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520a7ca379b34aef84d0343411d79e55" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36127 } } }
I20260811 07:39:40.097862 28942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 520a7ca379b34aef84d0343411d79e55. Latest consensus state: current_term: 1 leader_uuid: "520a7ca379b34aef84d0343411d79e55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520a7ca379b34aef84d0343411d79e55" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 36127 } } }
I20260811 07:39:40.098017 28941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:40.098017 28942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:40.099100 28952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:40.102063 28952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:40.108250 28952 catalog_manager.cc:1347] Generated new cluster ID: 13f8fcebb103482ba3ab35d2bce963b9
I20260811 07:39:40.108348 28952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:40.117012 28952 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:40.118345 28952 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:40.127492 28952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 520a7ca379b34aef84d0343411d79e55: Generated new TSK 0
I20260811 07:39:40.128248 28952 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:39:40.137202 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36127
--builtin_ntp_servers=127.18.72.212:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 07:39:40.277179 28962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:40.277428 28962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:40.277474 28962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:40.281596 28962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:40.281658 28962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:40.281718 28962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:40.287001 28962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:40.288244 28962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:40.289721 28962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:40.300020 28972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.300036 28968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.300035 28969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:40.300521 28962 server_base.cc:1034] running on GCE node
I20260811 07:39:40.300879 28962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:40.301586 28962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:40.302772 28962 hybrid_clock.cc:648] HybridClock initialized: now 1786433980302727 us; error 58 us; skew 500 ppm
I20260811 07:39:40.304621 28962 webserver.cc:460] Webserver started at http://127.18.72.193:36349/ using document root <none> and password file <none>
I20260811 07:39:40.305248 28962 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:40.305341 28962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:40.305576 28962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:40.307675 28962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "bfcc185d4ad6417ea75d2f77212c1377"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "4792a5feeeba49ec9860eb24e59203c1"
server_key_iv: "d38394a1ab319740056dceab8ebec312"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.308342 28962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "bfcc185d4ad6417ea75d2f77212c1377"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "4792a5feeeba49ec9860eb24e59203c1"
server_key_iv: "d38394a1ab319740056dceab8ebec312"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.312237 28962 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 07:39:40.314903 28978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.316233 28962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:40.316365 28962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "bfcc185d4ad6417ea75d2f77212c1377"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "4792a5feeeba49ec9860eb24e59203c1"
server_key_iv: "d38394a1ab319740056dceab8ebec312"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.316486 28962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:40.331368 28962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:40.332134 28962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:40.332374 28962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:40.332981 28962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:40.333967 28962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:40.334038 28962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.334111 28962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:40.334152 28962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.342773 28962 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:43513
I20260811 07:39:40.342803 29102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:43513 every 8 connection(s)
I20260811 07:39:40.343709 28962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:40.346167 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 28962
I20260811 07:39:40.346292 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:40.346638 18723 external_mini_cluster.cc:1428] Setting key 6db88fd4c49063c6b24ac10ecfb829eb
I20260811 07:39:40.348908 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36127
--builtin_ntp_servers=127.18.72.212:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 07:39:40.354218 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:40.345603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:55827 (local address 127.18.72.254:36127)
0811 07:39:40.345755 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:40.345758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:40.347040 (+ 1282us) server_negotiation.cc:407] Connection header received
0811 07:39:40.348824 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:40.348827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:40.348876 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:40.348952 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:40.350204 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.350769 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.352025 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.352257 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.352368 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:40.353624 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:40.353629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:40.353634 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:40.353651 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:40.353678 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:40.353695 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:40.353773 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:40.353961 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 07:39:40.354050 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:39:40.355123 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:40.346000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36127 (local address 127.18.72.193:55827)
0811 07:39:40.346846 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:40.346904 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:40.348576 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:40.349172 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:40.349184 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:40.349298 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:40.350032 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:40.350055 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.350917 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.350921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:40.351829 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:40.351840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.352418 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.352423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:40.352592 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:40.353285 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:40.353287 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:40.353329 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:40.353398 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:40.353775 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:40.353788 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:40.353795 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:39:40.353931 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 07:39:40.354234 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":156,"threads_started":1}
I20260811 07:39:40.356601 29103 heartbeater.cc:347] Connected to a master server at 127.18.72.254:36127
I20260811 07:39:40.356904 29103 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:40.357647 29103 heartbeater.cc:511] Master 127.18.72.254:36127 requested a full tablet report, sending...
I20260811 07:39:40.359370 28895 ts_manager.cc:194] Registered new tserver with Master: bfcc185d4ad6417ea75d2f77212c1377 (127.18.72.193:43513)
I20260811 07:39:40.360653 28895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.193:55827
W20260811 07:39:40.467427 29107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:40.467684 29107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:40.467722 29107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:40.471731 29107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:40.471791 29107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:40.471846 29107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:40.476688 29107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:40.477810 29107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:40.479245 29107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:40.488965 29117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.488971 29115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.488957 29114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:40.489197 29107 server_base.cc:1034] running on GCE node
I20260811 07:39:40.489688 29107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:40.490335 29107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:40.491526 29107 hybrid_clock.cc:648] HybridClock initialized: now 1786433980491508 us; error 28 us; skew 500 ppm
I20260811 07:39:40.493431 29107 webserver.cc:460] Webserver started at http://127.18.72.194:40421/ using document root <none> and password file <none>
I20260811 07:39:40.494059 29107 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:40.494153 29107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:40.494400 29107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:40.496618 29107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "67812c132867472dadc33381981406af"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "957bf6f40953dafbb4023efe81b23b21"
server_key_iv: "3298f7be2f801b64b254febf1525d47b"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.497179 29107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "67812c132867472dadc33381981406af"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "957bf6f40953dafbb4023efe81b23b21"
server_key_iv: "3298f7be2f801b64b254febf1525d47b"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.500962 29107 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:39:40.503400 29123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.504652 29107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:40.504796 29107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "67812c132867472dadc33381981406af"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "957bf6f40953dafbb4023efe81b23b21"
server_key_iv: "3298f7be2f801b64b254febf1525d47b"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.504909 29107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:40.517318 29107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:40.518021 29107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:40.518217 29107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:40.518810 29107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:40.519835 29107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:40.519942 29107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.520025 29107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:40.520072 29107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.529474 29107 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:46059
I20260811 07:39:40.529500 29244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:46059 every 8 connection(s)
I20260811 07:39:40.530413 29107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:40.536688 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 29107
I20260811 07:39:40.536845 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:40.537143 18723 external_mini_cluster.cc:1428] Setting key bf51dcde2379f0d19e2814d4ab98110b
I20260811 07:39:40.539166 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36127
--builtin_ntp_servers=127.18.72.212:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 07:39:40.540386 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:40.532383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:46875 (local address 127.18.72.254:36127)
0811 07:39:40.532567 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:40.532576 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:40.533765 (+ 1189us) server_negotiation.cc:407] Connection header received
0811 07:39:40.535125 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:40.535127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:40.535176 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:40.535257 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:40.536221 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.536830 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.538119 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.538369 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.538470 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:40.539735 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:40.539742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:40.539748 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:40.539766 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:40.539791 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:40.539804 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:40.539884 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:40.540092 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 07:39:40.540164 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 07:39:40.541162 29247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:40.532917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36127 (local address 127.18.72.194:46875)
0811 07:39:40.533590 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:40.533633 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:40.534925 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:40.535425 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:40.535432 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:40.535513 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:40.536056 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:40.536071 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.536959 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.536963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:40.538001 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:40.538013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.538535 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.538539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:40.538741 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:40.539470 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:40.539471 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:40.539512 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:40.539576 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:40.539944 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:40.539952 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:40.539957 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:39:40.540058 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 07:39:40.540304 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260811 07:39:40.542335 29245 heartbeater.cc:347] Connected to a master server at 127.18.72.254:36127
I20260811 07:39:40.542582 29245 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:40.543067 29245 heartbeater.cc:511] Master 127.18.72.254:36127 requested a full tablet report, sending...
I20260811 07:39:40.544152 28895 ts_manager.cc:194] Registered new tserver with Master: 67812c132867472dadc33381981406af (127.18.72.194:46059)
I20260811 07:39:40.545009 28895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.194:46875
W20260811 07:39:40.659108 29248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:40.659381 29248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:40.659421 29248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:40.663355 29248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:40.663407 29248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:40.663460 29248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:40.668057 29248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:36127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:40.669075 29248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:40.670433 29248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:40.680147 29254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.680222 29257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.680287 29255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:40.680162 29248 server_base.cc:1034] running on GCE node
I20260811 07:39:40.680872 29248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:40.681535 29248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:40.682718 29248 hybrid_clock.cc:648] HybridClock initialized: now 1786433980682693 us; error 30 us; skew 500 ppm
I20260811 07:39:40.684796 29248 webserver.cc:460] Webserver started at http://127.18.72.195:40629/ using document root <none> and password file <none>
I20260811 07:39:40.685520 29248 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:40.685593 29248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:40.685846 29248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:40.688033 29248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "958de020b4094b80bc4312895f42b479"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "3a5dc621892ad1be2ac878164ca1685a"
server_key_iv: "0518b5d0e2ca71a08ec4864d5433b088"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.688622 29248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "958de020b4094b80bc4312895f42b479"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "3a5dc621892ad1be2ac878164ca1685a"
server_key_iv: "0518b5d0e2ca71a08ec4864d5433b088"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.692485 29248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 07:39:40.695043 29264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.696410 29248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:40.696548 29248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "958de020b4094b80bc4312895f42b479"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "3a5dc621892ad1be2ac878164ca1685a"
server_key_iv: "0518b5d0e2ca71a08ec4864d5433b088"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.696658 29248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:40.713739 29248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:40.714484 29248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:40.714679 29248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:40.715289 29248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:40.716317 29248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:40.716365 29248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.716428 29248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:40.716465 29248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.725579 29248 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:35401
I20260811 07:39:40.725605 29387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:35401 every 8 connection(s)
I20260811 07:39:40.726495 29248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:40.728271 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 29248
I20260811 07:39:40.728389 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:40.728684 18723 external_mini_cluster.cc:1428] Setting key 1077ec0ba300fb9400e2523c668b4270
I20260811 07:39:40.736042 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:40.728406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:40621 (local address 127.18.72.254:36127)
0811 07:39:40.728556 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:40.728560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:40.729863 (+ 1303us) server_negotiation.cc:407] Connection header received
0811 07:39:40.731419 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:40.731422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:40.731470 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:40.731545 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:40.732469 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.733035 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.734214 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.734435 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.734517 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:40.735540 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:40.735543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:40.735545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:40.735557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:40.735577 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:40.735584 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:40.735654 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:40.735789 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 07:39:40.735841 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:39:40.736891 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:40.728913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:36127 (local address 127.18.72.195:40621)
0811 07:39:40.729692 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:40.729745 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:40.731175 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:40.731683 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:40.731690 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:40.731775 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:40.732307 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:40.732321 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.733192 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.733197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:40.734104 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:40.734114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:40.734565 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:40.734568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:40.734710 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:40.735266 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:40.735267 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:40.735307 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:40.735380 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:40.735664 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:40.735671 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:40.735675 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:39:40.735769 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 07:39:40.736047 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":153,"threads_started":1}
I20260811 07:39:40.738070 29388 heartbeater.cc:347] Connected to a master server at 127.18.72.254:36127
I20260811 07:39:40.738317 29388 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:40.738847 29388 heartbeater.cc:511] Master 127.18.72.254:36127 requested a full tablet report, sending...
I20260811 07:39:40.739946 28895 ts_manager.cc:194] Registered new tserver with Master: 958de020b4094b80bc4312895f42b479 (127.18.72.195:35401)
I20260811 07:39:40.740522 28895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.195:40621
I20260811 07:39:40.742555 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 07:39:40.745431 28938 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:40.744398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44784 (local address 127.18.72.254:36127)
0811 07:39:40.744550 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:40.744554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:40.744702 (+ 148us) server_negotiation.cc:407] Connection header received
0811 07:39:40.744827 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:40.744868 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:40.744907 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:40.744972 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:40.745341 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44784: BlockingRecv error: recv got EOF from 127.0.0.1:44784 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:39:40.745569 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 28962
I20260811 07:39:40.759342 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 29107
I20260811 07:39:40.770445 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 29248
I20260811 07:39:40.781101 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 28863
2026-08-11T07:39:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (915 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T07:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:40Z Disabled control of system clock
I20260811 07:39:40.813799 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:33041
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:33041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 07:39:40.926517 29400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:40.926913 29400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:40.927027 29400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:40.931438 29400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:40.931500 29400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:40.931516 29400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:40.931533 29400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:40.931582 29400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:40.936599 29400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:33041
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:33041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:40.937697 29400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:40.939136 29400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:39:40.947934 29400 server_base.cc:1034] running on GCE node
W20260811 07:39:40.947777 29405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.947780 29408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:40.947824 29406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:40.948729 29400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:40.949815 29400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:40.951013 29400 hybrid_clock.cc:648] HybridClock initialized: now 1786433980951002 us; error 28 us; skew 500 ppm
I20260811 07:39:40.952792 29400 webserver.cc:460] Webserver started at http://127.18.72.254:38017/ using document root <none> and password file <none>
I20260811 07:39:40.953369 29400 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:40.953457 29400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:40.953682 29400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:40.955715 29400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "7c8a1d1e35f74b34be4b8aa08e24a429"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "71fa70b22ecb4a571139287a73a7530b"
server_key_iv: "a48c1a941fd3a9b277077e959e111cc2"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.956341 29400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "7c8a1d1e35f74b34be4b8aa08e24a429"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "71fa70b22ecb4a571139287a73a7530b"
server_key_iv: "a48c1a941fd3a9b277077e959e111cc2"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.960099 29400 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:39:40.962435 29414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:40.963629 29400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:40.963764 29400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "7c8a1d1e35f74b34be4b8aa08e24a429"
format_stamp: "Formatted at 2026-08-11 07:39:40 on dist-test-slave-zkpd"
server_key: "71fa70b22ecb4a571139287a73a7530b"
server_key_iv: "a48c1a941fd3a9b277077e959e111cc2"
server_key_version: "encryptionkey@0"
I20260811 07:39:40.963892 29400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:40.968515 29400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:40.969170 29400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:40.969349 29400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:40.976156 29400 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:33041
I20260811 07:39:40.976207 29472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:33041 every 8 connection(s)
I20260811 07:39:40.977108 29400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:40.979176 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 29400
I20260811 07:39:40.979316 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:40.979605 18723 external_mini_cluster.cc:1428] Setting key 5bd05a9804e1607d3b130250598d7921
I20260811 07:39:40.981019 29473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:40.986382 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:40.981157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.18.72.254:33041)
0811 07:39:40.981894 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:40.981918 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:40.981965 (+ 47us) server_negotiation.cc:407] Connection header received
0811 07:39:40.982942 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:40.982968 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:40.983451 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:40.983865 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:40.984681 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:40.984687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:40.984703 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:40.984717 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:40.984806 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:40.984833 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:40.985001 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:40.985120 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 07:39:40.985399 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":112,"threads_started":1}
I20260811 07:39:40.988926 29473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429: Bootstrap starting.
I20260811 07:39:40.991238 29473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:40.992213 29473 log.cc:824] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:40.994216 29473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429: No bootstrap required, opened a new log
I20260811 07:39:40.998286 29473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a1d1e35f74b34be4b8aa08e24a429" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33041 } }
I20260811 07:39:40.998523 29473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:40.998569 29473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8a1d1e35f74b34be4b8aa08e24a429, State: Initialized, Role: FOLLOWER
I20260811 07:39:40.999022 29473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a1d1e35f74b34be4b8aa08e24a429" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33041 } }
I20260811 07:39:40.999154 29473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:40.999205 29473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:40.999292 29473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:41.000308 29473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a1d1e35f74b34be4b8aa08e24a429" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33041 } }
I20260811 07:39:41.000635 29473 leader_election.cc:302] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c8a1d1e35f74b34be4b8aa08e24a429; no voters:
I20260811 07:39:41.000917 29473 leader_election.cc:288] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:41.001039 29478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:41.001291 29478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [term 1 LEADER]: Becoming Leader. State: Replica: 7c8a1d1e35f74b34be4b8aa08e24a429, State: Running, Role: LEADER
I20260811 07:39:41.001646 29478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a1d1e35f74b34be4b8aa08e24a429" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33041 } }
I20260811 07:39:41.002254 29473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:41.003387 29479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c8a1d1e35f74b34be4b8aa08e24a429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a1d1e35f74b34be4b8aa08e24a429" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33041 } } }
I20260811 07:39:41.003542 29479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:41.003849 29480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c8a1d1e35f74b34be4b8aa08e24a429. Latest consensus state: current_term: 1 leader_uuid: "7c8a1d1e35f74b34be4b8aa08e24a429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a1d1e35f74b34be4b8aa08e24a429" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33041 } } }
I20260811 07:39:41.003978 29480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:41.003970 29487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:41.007023 29487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:41.013067 29487 catalog_manager.cc:1347] Generated new cluster ID: 4acb90ed6fb54e41bd7086f429e696d0
I20260811 07:39:41.013139 29487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:41.035789 29487 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:41.036170 29487 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:41.040714 29487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c8a1d1e35f74b34be4b8aa08e24a429: Generated new TSK 0
I20260811 07:39:41.041368 29487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:39:41.050622 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33041
--builtin_ntp_servers=127.18.72.212:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 07:39:41.164656 29498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:41.164922 29498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:41.164991 29498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:41.169272 29498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:41.169330 29498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:41.169418 29498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:41.174235 29498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:41.175336 29498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:41.176782 29498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:41.186667 29505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:41.186766 29498 server_base.cc:1034] running on GCE node
W20260811 07:39:41.186666 29504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:41.186693 29507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:41.187371 29498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:41.188001 29498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:41.189182 29498 hybrid_clock.cc:648] HybridClock initialized: now 1786433981189167 us; error 23 us; skew 500 ppm
I20260811 07:39:41.191102 29498 webserver.cc:460] Webserver started at http://127.18.72.193:38053/ using document root <none> and password file <none>
I20260811 07:39:41.191731 29498 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:41.191819 29498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:41.192073 29498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:41.194146 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "6f5de14a4fda47068f0337f539a4071f"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "0d0f7ab865b8fa63bcac3a0facdac07a"
server_key_iv: "149fda1d91af4d5884f643e316cad213"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.194732 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "6f5de14a4fda47068f0337f539a4071f"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "0d0f7ab865b8fa63bcac3a0facdac07a"
server_key_iv: "149fda1d91af4d5884f643e316cad213"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.198963 29498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:39:41.201542 29513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.202800 29498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:41.202916 29498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "6f5de14a4fda47068f0337f539a4071f"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "0d0f7ab865b8fa63bcac3a0facdac07a"
server_key_iv: "149fda1d91af4d5884f643e316cad213"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.203047 29498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:41.207551 29498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:41.208205 29498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:41.208381 29498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:41.208909 29498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:41.209789 29498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:41.209836 29498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.209908 29498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:41.209941 29498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.218250 29498 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:36733
I20260811 07:39:41.218272 29633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:36733 every 8 connection(s)
I20260811 07:39:41.219116 29498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:41.224977 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:41.220948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:42623 (local address 127.18.72.254:33041)
0811 07:39:41.221098 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:41.221103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:41.222150 (+ 1047us) server_negotiation.cc:407] Connection header received
0811 07:39:41.223393 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:41.223396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:41.223433 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:41.223506 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:41.224604 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:41.224606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:41.224608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:41.224618 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:41.224636 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:41.224644 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:41.224714 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:41.224833 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 07:39:41.224872 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:39:41.225816 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:41.221367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33041 (local address 127.18.72.193:42623)
0811 07:39:41.222018 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:41.222056 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:41.223206 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:41.223659 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:41.223666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:41.223750 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:41.224369 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:41.224371 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:41.224410 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:41.224476 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:41.224714 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:41.224721 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:41.224726 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:39:41.224861 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 07:39:41.225072 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260811 07:39:41.227171 29634 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33041
I20260811 07:39:41.227406 29634 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:41.227701 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 29498
I20260811 07:39:41.227818 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:39:41.227939 29634 heartbeater.cc:511] Master 127.18.72.254:33041 requested a full tablet report, sending...
I20260811 07:39:41.228189 18723 external_mini_cluster.cc:1428] Setting key 272550924f92d0499686102586f0ea50
I20260811 07:39:41.229645 29432 ts_manager.cc:194] Registered new tserver with Master: 6f5de14a4fda47068f0337f539a4071f (127.18.72.193:36733)
I20260811 07:39:41.230110 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33041
--builtin_ntp_servers=127.18.72.212:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 07:39:41.344581 29638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:41.344906 29638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:41.344986 29638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:41.349521 29638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:41.349594 29638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:41.349675 29638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:41.354405 29638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:41.355477 29638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:41.356953 29638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:41.366492 29647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:41.366578 29638 server_base.cc:1034] running on GCE node
W20260811 07:39:41.366479 29645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:41.366492 29644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:41.367274 29638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:41.368005 29638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:41.369179 29638 hybrid_clock.cc:648] HybridClock initialized: now 1786433981369120 us; error 53 us; skew 500 ppm
I20260811 07:39:41.371037 29638 webserver.cc:460] Webserver started at http://127.18.72.194:40873/ using document root <none> and password file <none>
I20260811 07:39:41.371616 29638 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:41.371677 29638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:41.371969 29638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:41.374017 29638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "7b819db1844046d2ac2e95953f31ced0"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "2ae43ec19a63683286d38f86d7bdcd4f"
server_key_iv: "7f245c99f0593b951fa6644e82c132dc"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.374580 29638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "7b819db1844046d2ac2e95953f31ced0"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "2ae43ec19a63683286d38f86d7bdcd4f"
server_key_iv: "7f245c99f0593b951fa6644e82c132dc"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.378310 29638 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 07:39:41.380788 29654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.381942 29638 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260811 07:39:41.382062 29638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "7b819db1844046d2ac2e95953f31ced0"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "2ae43ec19a63683286d38f86d7bdcd4f"
server_key_iv: "7f245c99f0593b951fa6644e82c132dc"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.382164 29638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:41.386595 29638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:41.387187 29638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:41.387362 29638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:41.387915 29638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:41.388816 29638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:41.388871 29638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.388933 29638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:41.388973 29638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.397464 29638 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:39169
I20260811 07:39:41.397503 29778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:39169 every 8 connection(s)
I20260811 07:39:41.398322 29638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:41.404284 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:41.400061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:60951 (local address 127.18.72.254:33041)
0811 07:39:41.400200 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:41.400204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:41.401215 (+ 1011us) server_negotiation.cc:407] Connection header received
0811 07:39:41.402587 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:41.402590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:41.402629 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:41.402703 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:41.403839 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:41.403842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:41.403844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:41.403856 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:41.403875 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:41.403883 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:41.404004 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:41.404123 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 07:39:41.404162 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:39:41.405203 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:41.400455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33041 (local address 127.18.72.194:60951)
0811 07:39:41.401080 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:41.401118 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:41.402377 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:41.402858 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:41.402864 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:41.402950 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:41.403573 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:41.403575 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:41.403631 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:41.403697 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:41.404003 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:41.404011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:41.404016 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:39:41.404144 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 07:39:41.404412 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260811 07:39:41.406304 29779 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33041
I20260811 07:39:41.406520 29779 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:41.406880 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 29638
I20260811 07:39:41.406950 29779 heartbeater.cc:511] Master 127.18.72.254:33041 requested a full tablet report, sending...
I20260811 07:39:41.406992 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:41.407263 18723 external_mini_cluster.cc:1428] Setting key 00ce14ebb0494218acf9a5acfd97e765
I20260811 07:39:41.408056 29432 ts_manager.cc:194] Registered new tserver with Master: 7b819db1844046d2ac2e95953f31ced0 (127.18.72.194:39169)
I20260811 07:39:41.409024 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33041
--builtin_ntp_servers=127.18.72.212:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 07:39:41.525316 29782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:41.525584 29782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:41.525652 29782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:41.529572 29782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:41.529640 29782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:41.529732 29782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:41.534287 29782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:41.535368 29782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:41.536687 29782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:41.546201 29789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:41.546279 29788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:41.546361 29782 server_base.cc:1034] running on GCE node
W20260811 07:39:41.546289 29791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:41.546983 29782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:41.547618 29782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:41.548796 29782 hybrid_clock.cc:648] HybridClock initialized: now 1786433981548758 us; error 52 us; skew 500 ppm
I20260811 07:39:41.550539 29782 webserver.cc:460] Webserver started at http://127.18.72.195:44569/ using document root <none> and password file <none>
I20260811 07:39:41.551120 29782 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:41.551208 29782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:41.551438 29782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:41.553547 29782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "a61d4040f7954049b53647378c97e33e"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "808fdf9c6ee028c69b9a232c8d555c33"
server_key_iv: "6a49c1bca6cbb2cd5bc52e8e6b0f84ab"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.554127 29782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "a61d4040f7954049b53647378c97e33e"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "808fdf9c6ee028c69b9a232c8d555c33"
server_key_iv: "6a49c1bca6cbb2cd5bc52e8e6b0f84ab"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.557811 29782 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:39:41.560290 29798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.561429 29782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:41.561558 29782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "a61d4040f7954049b53647378c97e33e"
format_stamp: "Formatted at 2026-08-11 07:39:41 on dist-test-slave-zkpd"
server_key: "808fdf9c6ee028c69b9a232c8d555c33"
server_key_iv: "6a49c1bca6cbb2cd5bc52e8e6b0f84ab"
server_key_version: "encryptionkey@0"
I20260811 07:39:41.561668 29782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:41.566922 29782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:41.567543 29782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:41.567750 29782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:41.568321 29782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:41.569213 29782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:41.569260 29782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.569326 29782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:41.569362 29782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:41.577795 29782 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:39223
I20260811 07:39:41.577853 29916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:39223 every 8 connection(s)
I20260811 07:39:41.578702 29782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:41.584614 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:41.580548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:37819 (local address 127.18.72.254:33041)
0811 07:39:41.580714 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:41.580719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:41.581779 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 07:39:41.583028 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:41.583031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:41.583068 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:41.583142 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:41.584230 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:41.584233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:41.584235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:39:41.584247 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:41.584268 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:39:41.584275 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:41.584352 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:41.584468 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 07:39:41.584506 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 07:39:41.585487 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:41.580996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33041 (local address 127.18.72.195:37819)
0811 07:39:41.581639 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:41.581678 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:41.582828 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:41.583298 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:41.583304 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:41.583407 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:41.583983 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:39:41.583985 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:41.584025 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:39:41.584094 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:41.584349 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:41.584357 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:41.584361 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 07:39:41.584489 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 07:39:41.584674 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260811 07:39:41.586112 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 29782
I20260811 07:39:41.586229 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:41.586570 18723 external_mini_cluster.cc:1428] Setting key aaa5f5b644ca02ecb1b00906a77f7619
I20260811 07:39:41.586681 29918 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33041
I20260811 07:39:41.586932 29918 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:41.587416 29918 heartbeater.cc:511] Master 127.18.72.254:33041 requested a full tablet report, sending...
I20260811 07:39:41.588550 29433 ts_manager.cc:194] Registered new tserver with Master: a61d4040f7954049b53647378c97e33e (127.18.72.195:39223)
I20260811 07:39:41.600404 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 07:39:41.603365 29476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:39:41.602298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.18.72.254:33041)
0811 07:39:41.602452 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:41.602458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:41.602600 (+ 142us) server_negotiation.cc:407] Connection header received
0811 07:39:41.602870 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:41.602872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:41.602907 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:41.602997 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:41.603230 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52138: BlockingRecv error: recv got EOF from 127.0.0.1:52138 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 07:39:41.603524 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 29498
I20260811 07:39:41.614248 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 29638
I20260811 07:39:41.622315 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 29782
I20260811 07:39:41.629890 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 29400
2026-08-11T07:39:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (847 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:41 dist-test-slave-zkpd krb5kdc[29932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:41 dist-test-slave-zkpd krb5kdc[29932](info): set up 2 sockets
Aug 11 07:39:41 dist-test-slave-zkpd krb5kdc[29932](info): commencing operation
krb5kdc: starting...
W20260811 07:39:42.713752 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:42 dist-test-slave-zkpd krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:39:42.836459 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41321
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:41321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:42.950109 29946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:42.950354 29946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:42.950425 29946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:42.954497 29946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:39:42.954556 29946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:42.954576 29946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:42.954592 29946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:39:42.954644 29946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:39:42.959810 29946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:41321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:41321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:42.960940 29946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:42.962013 29946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:42.970726 29954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:42.970721 29946 server_base.cc:1034] running on GCE node
W20260811 07:39:42.970718 29952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:42.970741 29956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:42.971496 29946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:42.972620 29946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:42.973778 29946 hybrid_clock.cc:648] HybridClock initialized: now 1786433982973764 us; error 25 us; skew 500 ppm
Aug 11 07:39:42 dist-test-slave-zkpd krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433982, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:42.981256 29946 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:39:42.982211 29946 webserver.cc:460] Webserver started at http://127.18.72.254:33065/ using document root <none> and password file <none>
I20260811 07:39:42.982785 29946 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:42.982867 29946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:42.983079 29946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:42.985200 29946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "64278b9469e54c3a90a2da9c144b0767"
format_stamp: "Formatted at 2026-08-11 07:39:42 on dist-test-slave-zkpd"
server_key: "e8dc340aadf0c3cebc05ebc8e0fed3f1"
server_key_iv: "29bb2de71e7fcff4f7715c86a14a2af2"
server_key_version: "encryptionkey@0"
I20260811 07:39:42.985766 29946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "64278b9469e54c3a90a2da9c144b0767"
format_stamp: "Formatted at 2026-08-11 07:39:42 on dist-test-slave-zkpd"
server_key: "e8dc340aadf0c3cebc05ebc8e0fed3f1"
server_key_iv: "29bb2de71e7fcff4f7715c86a14a2af2"
server_key_version: "encryptionkey@0"
I20260811 07:39:42.989566 29946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 07:39:42.992027 29963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:42.993247 29946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:39:42.993383 29946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "64278b9469e54c3a90a2da9c144b0767"
format_stamp: "Formatted at 2026-08-11 07:39:42 on dist-test-slave-zkpd"
server_key: "e8dc340aadf0c3cebc05ebc8e0fed3f1"
server_key_iv: "29bb2de71e7fcff4f7715c86a14a2af2"
server_key_version: "encryptionkey@0"
I20260811 07:39:42.993486 29946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:43.011493 29946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:43.014747 29946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:43.014967 29946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:43.022297 29946 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:41321
I20260811 07:39:43.022583 30017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:41321 every 8 connection(s)
I20260811 07:39:43.023295 29946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:39:43.026180 30018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:43.031631 30018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767: Bootstrap starting.
I20260811 07:39:43.033109 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 29946
I20260811 07:39:43.033238 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:39:43.033535 18723 external_mini_cluster.cc:1428] Setting key c2f61e2087dae9e4962fc1e2cad4f9db
I20260811 07:39:43.034612 30018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:43.035444 30018 log.cc:824] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:43.037832 30018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767: No bootstrap required, opened a new log
I20260811 07:39:43.042325 30018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64278b9469e54c3a90a2da9c144b0767" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41321 } }
I20260811 07:39:43.042550 30018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:43.042604 30018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64278b9469e54c3a90a2da9c144b0767, State: Initialized, Role: FOLLOWER
Aug 11 07:39:43 dist-test-slave-zkpd krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:43.043100 30018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64278b9469e54c3a90a2da9c144b0767" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41321 } }
I20260811 07:39:43.043248 30018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:39:43.043335 30018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:39:43.043440 30018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:43.044747 30018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64278b9469e54c3a90a2da9c144b0767" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41321 } }
I20260811 07:39:43.045121 30018 leader_election.cc:302] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64278b9469e54c3a90a2da9c144b0767; no voters:
I20260811 07:39:43.045470 30018 leader_election.cc:288] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:39:43.045573 30023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:43.045823 30023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [term 1 LEADER]: Becoming Leader. State: Replica: 64278b9469e54c3a90a2da9c144b0767, State: Running, Role: LEADER
I20260811 07:39:43.046170 30023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64278b9469e54c3a90a2da9c144b0767" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41321 } }
I20260811 07:39:43.046855 30018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:39:43.047796 30025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64278b9469e54c3a90a2da9c144b0767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64278b9469e54c3a90a2da9c144b0767" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41321 } } }
I20260811 07:39:43.047955 30025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:43.048291 30027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64278b9469e54c3a90a2da9c144b0767. Latest consensus state: current_term: 1 leader_uuid: "64278b9469e54c3a90a2da9c144b0767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64278b9469e54c3a90a2da9c144b0767" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 41321 } } }
I20260811 07:39:43.048380 30027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767 [sys.catalog]: This master's current role is: LEADER
I20260811 07:39:43.049623 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.034713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58272 (local address 127.18.72.254:41321)
0811 07:39:43.035177 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.035194 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:43.035230 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:39:43.036034 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:43.036062 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:43.036458 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:43.036739 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:43.037664 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.038756 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.039949 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.040422 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.040536 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.043547 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:43.043554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:43.043571 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:43.043603 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:43.045618 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.046159 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.046164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.046171 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.046334 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.046658 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.046661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.046663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.047174 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:43.048150 (+ 976us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:43.048303 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:39:43.048637 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":142,"threads_started":1}
I20260811 07:39:43.049381 30034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:39:43.052486 30034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:39:43.058230 30034 catalog_manager.cc:1347] Generated new cluster ID: 1d26a2c76afb4a1d869e6547b31c85a5
I20260811 07:39:43.058320 30034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:39:43.076097 30034 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:39:43.077009 30034 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:39:43.092345 30034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767: Generated new TSK 0
I20260811 07:39:43.093071 30034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:39:43.130800 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41321
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:43.246408 30047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:43.246786 30047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:43.246863 30047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:43.251133 30047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:43.251199 30047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:43.251305 30047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:39:43.256616 30047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:43.257781 30047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:43.258848 30047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:43.268704 30052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:43.268769 30047 server_base.cc:1034] running on GCE node
W20260811 07:39:43.268708 30053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:43.268893 30055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:43.269443 30047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:43.270023 30047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:43.271189 30047 hybrid_clock.cc:648] HybridClock initialized: now 1786433983271161 us; error 38 us; skew 500 ppm
Aug 11 07:39:43 dist-test-slave-zkpd krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:43.274224 30047 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:39:43.275187 30047 webserver.cc:460] Webserver started at http://127.18.72.193:33501/ using document root <none> and password file <none>
I20260811 07:39:43.275764 30047 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:43.275835 30047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:43.276096 30047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:43.278159 30047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "aaa9d386d6f347f180f99f4400327882"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "02c3de7bc69532b91f863f6b06774f84"
server_key_iv: "f44cfd48a4a28a212ed01b84d015f147"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.278703 30047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "aaa9d386d6f347f180f99f4400327882"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "02c3de7bc69532b91f863f6b06774f84"
server_key_iv: "f44cfd48a4a28a212ed01b84d015f147"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.282444 30047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:39:43.284920 30064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.286149 30047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:43.286271 30047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "aaa9d386d6f347f180f99f4400327882"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "02c3de7bc69532b91f863f6b06774f84"
server_key_iv: "f44cfd48a4a28a212ed01b84d015f147"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.286375 30047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:43.304643 30047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:43.307454 30047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:43.307607 30047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:43.308199 30047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:43.308988 30047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:43.309042 30047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.309082 30047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:43.309104 30047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.317952 30047 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:36185
I20260811 07:39:43.317976 30186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:36185 every 8 connection(s)
I20260811 07:39:43.318849 30047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:39:43.328085 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 30047
I20260811 07:39:43.328215 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
Aug 11 07:39:43 dist-test-slave-zkpd krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:43.328501 18723 external_mini_cluster.cc:1428] Setting key 28e9f451ecbf189335ac15412c5d65ae
I20260811 07:39:43.332770 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.320628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:45291 (local address 127.18.72.254:41321)
0811 07:39:43.320772 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.320777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:43.321660 (+ 883us) server_negotiation.cc:407] Connection header received
0811 07:39:43.323005 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:43.323007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:43.323055 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:43.323141 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:43.324396 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.324930 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.326047 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.326285 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.326369 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.328864 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:43.328868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:43.328871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:43.328897 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:43.330446 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.331054 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.331056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.331058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.331130 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.331521 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.331524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.331525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.331686 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:43.331756 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:43.332410 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 07:39:43.332555 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:39:43.333534 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.321056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41321 (local address 127.18.72.193:45291)
0811 07:39:43.321515 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:43.321558 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:43.322791 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:43.323301 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:43.323308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:43.323663 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:43.324225 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.324241 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.325051 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.325054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.325932 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.325942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.326416 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.326419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.326555 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.327136 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:43.327153 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:43.328706 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:43.330558 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:43.330567 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:43.330584 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:43.330959 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:43.331216 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:43.331222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:43.331227 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:43.331424 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:43.331781 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:43.331790 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:43.332143 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 07:39:43.332307 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 07:39:43.332553 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260811 07:39:43.334860 30187 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41321
I20260811 07:39:43.335136 30187 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:43.335754 30187 heartbeater.cc:511] Master 127.18.72.254:41321 requested a full tablet report, sending...
I20260811 07:39:43.337381 29981 ts_manager.cc:194] Registered new tserver with Master: aaa9d386d6f347f180f99f4400327882 (127.18.72.193:36185)
I20260811 07:39:43.338584 29981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:45291
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:39:43.356899 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41321
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:43.472425 30193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:43.472674 30193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:43.472718 30193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:43.476670 30193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:43.476722 30193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:43.476778 30193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:39:43.481873 30193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:43.482831 30193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:43.483779 30193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:43.493744 30201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:43.493763 30199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:43.493835 30193 server_base.cc:1034] running on GCE node
W20260811 07:39:43.493708 30198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:43.494308 30193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:43.494882 30193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:43.496038 30193 hybrid_clock.cc:648] HybridClock initialized: now 1786433983496017 us; error 28 us; skew 500 ppm
Aug 11 07:39:43 dist-test-slave-zkpd krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:43.498883 30193 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:39:43.499689 30193 webserver.cc:460] Webserver started at http://127.18.72.194:33487/ using document root <none> and password file <none>
I20260811 07:39:43.500295 30193 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:43.500358 30193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:43.500559 30193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:43.502521 30193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "16c377e301aa4ec1aa7dac507beffb47"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "0c98e942b3d1f6ce9806c6c48b0833ef"
server_key_iv: "2873660788e965dd6397ec1a99abe4f9"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.503073 30193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "16c377e301aa4ec1aa7dac507beffb47"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "0c98e942b3d1f6ce9806c6c48b0833ef"
server_key_iv: "2873660788e965dd6397ec1a99abe4f9"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.506709 30193 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:39:43.509122 30210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.510288 30193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:39:43.510404 30193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "16c377e301aa4ec1aa7dac507beffb47"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "0c98e942b3d1f6ce9806c6c48b0833ef"
server_key_iv: "2873660788e965dd6397ec1a99abe4f9"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.510500 30193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:43.546831 30193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:43.550122 30193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:43.550315 30193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:43.550945 30193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:43.551820 30193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:43.551873 30193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.551981 30193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:43.552016 30193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.560698 30193 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:39653
I20260811 07:39:43.560726 30330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:39653 every 8 connection(s)
I20260811 07:39:43.561566 30193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:39:43.563849 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 30193
I20260811 07:39:43.564000 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:39:43.564272 18723 external_mini_cluster.cc:1428] Setting key 26b2c36899fbdce4b22ceceea12219c5
Aug 11 07:39:43 dist-test-slave-zkpd krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:39:43.576866 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.563301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:57293 (local address 127.18.72.254:41321)
0811 07:39:43.563430 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.563434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:43.564375 (+ 941us) server_negotiation.cc:407] Connection header received
0811 07:39:43.566066 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:43.566072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:43.566127 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:43.566209 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:43.567370 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.568150 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.569813 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.570203 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.570348 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.573233 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:43.573236 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:43.573238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:43.573262 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:43.574651 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.575266 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.575268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.575269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.575343 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.575714 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.575716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.575717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.576000 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:43.576079 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:43.576630 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 07:39:43.576728 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:39:43.577673 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.563666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41321 (local address 127.18.72.194:57293)
0811 07:39:43.564167 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:43.564219 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:43.565870 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:43.566385 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:43.566392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:43.566754 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:43.567228 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.567241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.568292 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.568297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.569679 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.569692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.570363 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.570365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.570498 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.571170 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:43.571185 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:43.573060 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:43.574767 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:43.574773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:43.574790 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:43.575167 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:43.575437 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:43.575442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:43.575445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:43.575629 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:43.576113 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:43.576121 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:43.576422 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 07:39:43.576587 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 07:39:43.576846 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":163,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
I20260811 07:39:43.578863 30331 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41321
I20260811 07:39:43.579106 30331 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:43.579591 30331 heartbeater.cc:511] Master 127.18.72.254:41321 requested a full tablet report, sending...
I20260811 07:39:43.580785 29981 ts_manager.cc:194] Registered new tserver with Master: 16c377e301aa4ec1aa7dac507beffb47 (127.18.72.194:39653)
I20260811 07:39:43.581332 29981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:57293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:39:43.594559 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41321
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:44501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:39:43.706763 30336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:39:43.706995 30336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:39:43.707034 30336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:39:43.710932 30336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:39:43.710981 30336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:39:43.711035 30336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:39:43.716193 30336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:44501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:41321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:39:43.717108 30336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:39:43.718037 30336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:39:43.727404 30342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:43.727406 30343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:39:43.727409 30345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:39:43.727468 30336 server_base.cc:1034] running on GCE node
I20260811 07:39:43.728040 30336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:39:43.728595 30336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:39:43.729744 30336 hybrid_clock.cc:648] HybridClock initialized: now 1786433983729718 us; error 44 us; skew 500 ppm
Aug 11 07:39:43 dist-test-slave-zkpd krb5kdc[29932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:39:43.732425 30336 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:39:43.733258 30336 webserver.cc:460] Webserver started at http://127.18.72.195:37807/ using document root <none> and password file <none>
I20260811 07:39:43.733816 30336 fs_manager.cc:359] Metadata directory not provided
I20260811 07:39:43.733906 30336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:39:43.734123 30336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:39:43.736181 30336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "c705663239214acfb94ee9a27772f8ee"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "c85ee225414d9448410ee4f65ae51a32"
server_key_iv: "321fd5e66063649c198f49f421a1ed1b"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.736727 30336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "c705663239214acfb94ee9a27772f8ee"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "c85ee225414d9448410ee4f65ae51a32"
server_key_iv: "321fd5e66063649c198f49f421a1ed1b"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.740360 30336 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:39:43.742687 30353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.743855 30336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:39:43.744038 30336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "c705663239214acfb94ee9a27772f8ee"
format_stamp: "Formatted at 2026-08-11 07:39:43 on dist-test-slave-zkpd"
server_key: "c85ee225414d9448410ee4f65ae51a32"
server_key_iv: "321fd5e66063649c198f49f421a1ed1b"
server_key_version: "encryptionkey@0"
I20260811 07:39:43.744144 30336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:39:43.755265 30336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:39:43.758078 30336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:39:43.758268 30336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:39:43.758839 30336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:39:43.759637 30336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:39:43.759702 30336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.759769 30336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:39:43.759814 30336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:39:43.768697 30336 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:46299
I20260811 07:39:43.768720 30477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:46299 every 8 connection(s)
I20260811 07:39:43.769526 30336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:39:43.770996 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 30336
I20260811 07:39:43.771117 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:39:43.771483 18723 external_mini_cluster.cc:1428] Setting key e274c80f6b67be626b24cedc70cf3018
Aug 11 07:39:43 dist-test-slave-zkpd krb5kdc[29932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:39:43.782572 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.771345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:40333 (local address 127.18.72.254:41321)
0811 07:39:43.771498 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.771503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:43.772538 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 07:39:43.774142 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:43.774144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:43.774188 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:43.774255 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:43.775360 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.775878 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.777004 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.777214 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.777293 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.779606 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:43.779609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:43.779611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:43.779634 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:43.780890 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.781323 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.781326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.781327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.781399 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.781671 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.781673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.781674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.781843 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:43.781939 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:43.782364 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 07:39:43.782457 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:39:43.783321 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.771791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:41321 (local address 127.18.72.195:40333)
0811 07:39:43.772396 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:43.772454 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:43.773961 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:43.774424 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:43.774431 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:43.774773 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:39:43.775233 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.775246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.776029 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.776032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.776893 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.776903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.777348 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.777351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.777490 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.778066 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:39:43.778081 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:39:43.779463 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:39:43.780989 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:43.780994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:43.781005 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:43.781241 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:43.781472 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:39:43.781474 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:39:43.781476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:39:43.781599 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:39:43.781947 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:39:43.781951 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:39:43.782156 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 07:39:43.782314 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 07:39:43.782546 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":149,"threads_started":1}
I20260811 07:39:43.784473 30478 heartbeater.cc:347] Connected to a master server at 127.18.72.254:41321
I20260811 07:39:43.784735 30478 heartbeater.cc:464] Registering TS with master...
I20260811 07:39:43.785301 30478 heartbeater.cc:511] Master 127.18.72.254:41321 requested a full tablet report, sending...
I20260811 07:39:43.786232 29981 ts_manager.cc:194] Registered new tserver with Master: c705663239214acfb94ee9a27772f8ee (127.18.72.195:46299)
I20260811 07:39:43.786732 29981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:40333
I20260811 07:39:43.796756 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:39:43.805773 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.798662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58288 (local address 127.18.72.254:41321)
0811 07:39:43.798799 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.798805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:43.799002 (+ 197us) server_negotiation.cc:407] Connection header received
0811 07:39:43.799150 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:43.799153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:43.799196 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:43.799285 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:39:43.799999 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.800534 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.801330 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.801559 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.801660 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.802925 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:39:43.802929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:39:43.802932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:39:43.802956 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:39:43.804148 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.804549 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.804553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.804555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.804635 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:39:43.804906 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:39:43.804910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:39:43.804912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:39:43.805127 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:39:43.805263 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:43.805496 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 07:39:43.805615 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:39:43.808609 29981 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:39:43.812434 29981 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:39:43.827941 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.822762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:39653 (local address 127.0.0.1:47038)
0811 07:39:43.823317 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:43.823331 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:43.823433 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:43.823943 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:43.823946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:43.823963 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:43.824081 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.824087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.825229 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.825231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.826215 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.826225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.827348 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.827352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.827466 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.827511 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 07:39:43.827615 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 07:39:43.827736 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":23,"thread_start_us":92,"threads_started":1}
I20260811 07:39:43.828375 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.822926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.18.72.194:39653)
0811 07:39:43.823264 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.823277 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:43.823534 (+ 257us) server_negotiation.cc:407] Connection header received
0811 07:39:43.823623 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:43.823631 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:43.823799 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:43.823979 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:43.824173 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.825067 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.826335 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.827221 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.827342 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.827481 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:43.828140 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 07:39:43.828235 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":109,"threads_started":1}
I20260811 07:39:43.828364 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.822130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.18.72.195:46299)
0811 07:39:43.822668 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.822683 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:43.822708 (+ 25us) server_negotiation.cc:407] Connection header received
0811 07:39:43.822792 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:43.822798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:43.823028 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:43.823198 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:43.823751 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.824961 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.826598 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.827498 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.827616 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.827750 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:43.827845 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 07:39:43.828008 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":158,"threads_started":1}
I20260811 07:39:43.828887 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.821985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:46299 (local address 127.0.0.1:54012)
0811 07:39:43.822319 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:43.822365 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:43.822490 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:43.823433 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:43.823439 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:43.823466 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:43.823604 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.823616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.825112 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.825115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.826467 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.826479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.827620 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.827624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.827721 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.827747 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:39:43.827818 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 07:39:43.827914 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":25,"thread_start_us":70,"threads_started":1}
I20260811 07:39:43.830180 30494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.823065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:36185 (local address 127.0.0.1:60374)
0811 07:39:43.823517 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:43.823530 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:43.823605 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:43.824370 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:43.824375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:43.824396 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:43.824473 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.824478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.826544 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.826548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.827638 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:43.827647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.829795 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.829799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:43.829901 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.829924 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 07:39:43.830003 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 07:39:43.830070 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":71,"threads_started":1}
I20260811 07:39:43.830289 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.823487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.18.72.193:36185)
0811 07:39:43.823905 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.823921 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:43.823941 (+ 20us) server_negotiation.cc:407] Connection header received
0811 07:39:43.824016 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:43.824020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:43.824201 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:43.824368 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:43.824570 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.826393 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.827785 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:43.829492 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:43.829597 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:43.829721 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:43.830055 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 07:39:43.830151 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":113,"threads_started":1}
I20260811 07:39:43.831401 30263 tablet_service.cc:1467] Processing CreateTablet for tablet 4657edf9930f4029a47a7211c4beab3b (DEFAULT_TABLE table=test-table [id=be299a3d617449678880be25e02e3c1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:43.831691 30405 tablet_service.cc:1467] Processing CreateTablet for tablet 4657edf9930f4029a47a7211c4beab3b (DEFAULT_TABLE table=test-table [id=be299a3d617449678880be25e02e3c1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:43.832499 30263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4657edf9930f4029a47a7211c4beab3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:43.832777 30405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4657edf9930f4029a47a7211c4beab3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:43.832814 30116 tablet_service.cc:1467] Processing CreateTablet for tablet 4657edf9930f4029a47a7211c4beab3b (DEFAULT_TABLE table=test-table [id=be299a3d617449678880be25e02e3c1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:39:43.833784 30116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4657edf9930f4029a47a7211c4beab3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:39:43.838795 30496 tablet_bootstrap.cc:492] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Bootstrap starting.
I20260811 07:39:43.840219 30497 tablet_bootstrap.cc:492] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Bootstrap starting.
I20260811 07:39:43.841810 30496 tablet_bootstrap.cc:654] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:43.842635 30497 tablet_bootstrap.cc:654] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:43.842681 30496 log.cc:824] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:43.843451 30498 tablet_bootstrap.cc:492] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882: Bootstrap starting.
I20260811 07:39:43.843954 30497 log.cc:824] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:43.844789 30496 tablet_bootstrap.cc:492] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: No bootstrap required, opened a new log
I20260811 07:39:43.844997 30496 ts_tablet_manager.cc:1397] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:39:43.845492 30498 tablet_bootstrap.cc:654] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882: Neither blocks nor log segments found. Creating new log.
I20260811 07:39:43.846007 30497 tablet_bootstrap.cc:492] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: No bootstrap required, opened a new log
I20260811 07:39:43.846222 30497 ts_tablet_manager.cc:1397] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 07:39:43.846297 30498 log.cc:824] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882: Log is configured to *not* fsync() on all Append() calls
I20260811 07:39:43.848318 30498 tablet_bootstrap.cc:492] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882: No bootstrap required, opened a new log
I20260811 07:39:43.848505 30498 ts_tablet_manager.cc:1397] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:39:43.848845 30496 raft_consensus.cc:348] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:43.849076 30496 raft_consensus.cc:374] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:43.849128 30496 raft_consensus.cc:729] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c377e301aa4ec1aa7dac507beffb47, State: Initialized, Role: FOLLOWER
I20260811 07:39:43.849529 30496 consensus_queue.cc:260] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:43.850463 30496 ts_tablet_manager.cc:1428] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 07:39:43.850523 30497 raft_consensus.cc:348] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:43.850778 30497 raft_consensus.cc:374] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:43.850838 30497 raft_consensus.cc:729] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c705663239214acfb94ee9a27772f8ee, State: Initialized, Role: FOLLOWER
I20260811 07:39:43.851254 30497 consensus_queue.cc:260] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:43.851877 30478 heartbeater.cc:503] Master 127.18.72.254:41321 was elected leader, sending a full tablet report...
I20260811 07:39:43.852066 30331 heartbeater.cc:503] Master 127.18.72.254:41321 was elected leader, sending a full tablet report...
I20260811 07:39:43.852237 30497 ts_tablet_manager.cc:1428] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 07:39:43.852746 30498 raft_consensus.cc:348] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:43.853006 30498 raft_consensus.cc:374] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:39:43.853076 30498 raft_consensus.cc:729] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa9d386d6f347f180f99f4400327882, State: Initialized, Role: FOLLOWER
I20260811 07:39:43.853518 30498 consensus_queue.cc:260] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:43.854267 30187 heartbeater.cc:503] Master 127.18.72.254:41321 was elected leader, sending a full tablet report...
I20260811 07:39:43.854506 30498 ts_tablet_manager.cc:1428] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:39:43.997618 30505 raft_consensus.cc:482] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:39:43.997890 30505 raft_consensus.cc:504] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:43.999264 30505 leader_election.cc:288] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c705663239214acfb94ee9a27772f8ee (127.18.72.195:46299), 16c377e301aa4ec1aa7dac507beffb47 (127.18.72.194:39653)
I20260811 07:39:44.003319 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.999617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:58651 (local address 127.18.72.195:46299)
0811 07:39:43.999784 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:43.999791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:44.000010 (+ 219us) server_negotiation.cc:407] Connection header received
0811 07:39:44.000257 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:44.000260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:44.000317 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:44.000373 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:44.000811 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.001309 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.002384 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.002782 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.002913 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:44.002969 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:44.003129 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 07:39:44.003195 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:39:44.003342 30506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.999513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:46299 (local address 127.18.72.193:58651)
0811 07:39:43.999872 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:43.999919 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:44.000107 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:44.000559 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:44.000562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:44.000582 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:44.000706 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:44.000711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.001428 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.001433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:44.002255 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:44.002264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.002903 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.002907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:44.003001 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:44.003030 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 07:39:44.003113 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 07:39:44.003182 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":1,"thread_start_us":145,"threads_started":1}
I20260811 07:39:44.003456 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:44.000175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:35527 (local address 127.18.72.194:39653)
0811 07:39:44.000302 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:44.000308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:44.000485 (+ 177us) server_negotiation.cc:407] Connection header received
0811 07:39:44.000641 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:44.000644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:44.000687 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:44.000769 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:39:44.001043 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.001519 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.002512 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.002906 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.003030 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:44.003082 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:44.003266 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 07:39:44.003337 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 07:39:44.003489 30508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:43.999992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:39653 (local address 127.18.72.193:35527)
0811 07:39:44.000366 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:44.000398 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 07:39:44.000511 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:39:44.000788 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:39:44.000791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:39:44.000807 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:39:44.000933 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:44.000938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.001631 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.001635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:44.002387 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:39:44.002395 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.003025 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.003029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:39:44.003138 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:44.003160 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:39:44.003250 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 07:39:44.003319 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260811 07:39:44.003983 30425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4657edf9930f4029a47a7211c4beab3b" candidate_uuid: "aaa9d386d6f347f180f99f4400327882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c705663239214acfb94ee9a27772f8ee" is_pre_election: true
I20260811 07:39:44.004097 30284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4657edf9930f4029a47a7211c4beab3b" candidate_uuid: "aaa9d386d6f347f180f99f4400327882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16c377e301aa4ec1aa7dac507beffb47" is_pre_election: true
I20260811 07:39:44.004323 30425 raft_consensus.cc:2454] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaa9d386d6f347f180f99f4400327882 in term 0.
I20260811 07:39:44.004326 30284 raft_consensus.cc:2454] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaa9d386d6f347f180f99f4400327882 in term 0.
I20260811 07:39:44.004797 30068 leader_election.cc:302] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa9d386d6f347f180f99f4400327882, c705663239214acfb94ee9a27772f8ee; no voters:
I20260811 07:39:44.005077 30505 raft_consensus.cc:2781] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:39:44.005172 30505 raft_consensus.cc:482] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:39:44.005249 30505 raft_consensus.cc:3037] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:44.006314 30505 raft_consensus.cc:504] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:44.006636 30505 leader_election.cc:288] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [CANDIDATE]: Term 1 election: Requested vote from peers c705663239214acfb94ee9a27772f8ee (127.18.72.195:46299), 16c377e301aa4ec1aa7dac507beffb47 (127.18.72.194:39653)
I20260811 07:39:44.007005 30425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4657edf9930f4029a47a7211c4beab3b" candidate_uuid: "aaa9d386d6f347f180f99f4400327882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c705663239214acfb94ee9a27772f8ee"
I20260811 07:39:44.007098 30284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4657edf9930f4029a47a7211c4beab3b" candidate_uuid: "aaa9d386d6f347f180f99f4400327882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16c377e301aa4ec1aa7dac507beffb47"
I20260811 07:39:44.007140 30425 raft_consensus.cc:3037] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:44.007249 30284 raft_consensus.cc:3037] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:39:44.008306 30425 raft_consensus.cc:2454] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaa9d386d6f347f180f99f4400327882 in term 1.
I20260811 07:39:44.008370 30284 raft_consensus.cc:2454] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaa9d386d6f347f180f99f4400327882 in term 1.
I20260811 07:39:44.008611 30068 leader_election.cc:302] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa9d386d6f347f180f99f4400327882, c705663239214acfb94ee9a27772f8ee; no voters:
I20260811 07:39:44.008780 30505 raft_consensus.cc:2781] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:39:44.008978 30505 raft_consensus.cc:686] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [term 1 LEADER]: Becoming Leader. State: Replica: aaa9d386d6f347f180f99f4400327882, State: Running, Role: LEADER
I20260811 07:39:44.009303 30505 consensus_queue.cc:237] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } }
I20260811 07:39:44.012460 29980 catalog_manager.cc:5495] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 reported cstate change: term changed from 0 to 1, leader changed from <none> to aaa9d386d6f347f180f99f4400327882 (127.18.72.193). New cstate: current_term: 1 leader_uuid: "aaa9d386d6f347f180f99f4400327882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaa9d386d6f347f180f99f4400327882" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 36185 } health_report { overall_health: HEALTHY } } }
W20260811 07:39:44.020411 30479 tablet.cc:2369] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:39:44.062708 30332 tablet.cc:2369] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:44.064465 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:44.060909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.18.72.193:36185)
0811 07:39:44.061047 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:39:44.061051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:39:44.061066 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:39:44.061167 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:39:44.061171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:39:44.061227 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:39:44.061348 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:39:44.061591 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.062092 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.063058 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:39:44.063270 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:39:44.063361 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:39:44.063639 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:39:44.064062 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:39:44.064152 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:39:44.064232 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 07:39:44.064282 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 07:39:44.070099 30188 tablet.cc:2369] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:39:44.070803 30425 raft_consensus.cc:1261] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 1 FOLLOWER]: Refusing update from remote peer aaa9d386d6f347f180f99f4400327882: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:44.070855 30284 raft_consensus.cc:1261] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 1 FOLLOWER]: Refusing update from remote peer aaa9d386d6f347f180f99f4400327882: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:39:44.071600 30509 consensus_queue.cc:1035] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c705663239214acfb94ee9a27772f8ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:44.071853 30505 consensus_queue.cc:1035] T 4657edf9930f4029a47a7211c4beab3b P aaa9d386d6f347f180f99f4400327882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16c377e301aa4ec1aa7dac507beffb47" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:39:44.080538 30513 mvcc.cc:205] Tried to move back new op lower bound from 7317233598744629248 to 7317233598504943616. Current Snapshot: MvccSnapshot[applied={T|T < 7317233598744629248 or (T in {7317233598744629248})}]
I20260811 07:39:44.087227 29981 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:44.087383 29981 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:39:44.089780 29981 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64278b9469e54c3a90a2da9c144b0767: Sending DeleteTablet for 3 replicas of tablet 4657edf9930f4029a47a7211c4beab3b
I20260811 07:39:44.090523 30405 tablet_service.cc:1514] Processing DeleteTablet for tablet 4657edf9930f4029a47a7211c4beab3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:44 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:54012
I20260811 07:39:44.090864 30116 tablet_service.cc:1514] Processing DeleteTablet for tablet 4657edf9930f4029a47a7211c4beab3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:44 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:60374
I20260811 07:39:44.091012 30522 tablet_replica.cc:330] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: stopping tablet replica
I20260811 07:39:44.091231 30263 tablet_service.cc:1514] Processing DeleteTablet for tablet 4657edf9930f4029a47a7211c4beab3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:39:44 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:47038
I20260811 07:39:44.091336 30522 raft_consensus.cc:2229] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:44.091552 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 30047
I20260811 07:39:44.091645 30522 raft_consensus.cc:2258] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:44.093699 30524 tablet_replica.cc:330] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: stopping tablet replica
I20260811 07:39:44.094074 30524 raft_consensus.cc:2229] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:39:44.094401 30524 raft_consensus.cc:2258] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:39:44.095712 30522 ts_tablet_manager.cc:1905] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:44.095834 30524 ts_tablet_manager.cc:1905] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:39:44.098356 30522 ts_tablet_manager.cc:1918] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:44.098467 30522 log.cc:1196] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/4657edf9930f4029a47a7211c4beab3b
I20260811 07:39:44.098779 30522 ts_tablet_manager.cc:1939] T 4657edf9930f4029a47a7211c4beab3b P c705663239214acfb94ee9a27772f8ee: Deleting consensus metadata
I20260811 07:39:44.099285 30524 ts_tablet_manager.cc:1918] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:39:44.099390 30524 log.cc:1196] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/4657edf9930f4029a47a7211c4beab3b
I20260811 07:39:44.099696 30524 ts_tablet_manager.cc:1939] T 4657edf9930f4029a47a7211c4beab3b P 16c377e301aa4ec1aa7dac507beffb47: Deleting consensus metadata
I20260811 07:39:44.099967 29966 catalog_manager.cc:4841] TS c705663239214acfb94ee9a27772f8ee (127.18.72.195:46299): tablet 4657edf9930f4029a47a7211c4beab3b (table test-table [id=be299a3d617449678880be25e02e3c1f]) successfully deleted
I20260811 07:39:44.100975 29966 catalog_manager.cc:4841] TS 16c377e301aa4ec1aa7dac507beffb47 (127.18.72.194:39653): tablet 4657edf9930f4029a47a7211c4beab3b (table test-table [id=be299a3d617449678880be25e02e3c1f]) successfully deleted
W20260811 07:39:44.103188 29966 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:36185 (error 108)
I20260811 07:39:44.103395 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 30193
I20260811 07:39:44.104497 30494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:39:44.104121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:36185 (local address 127.0.0.1:60382)
0811 07:39:44.104372 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 07:39:44.104429 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260811 07:39:44.104638 29966 catalog_manager.cc:4584] TS aaa9d386d6f347f180f99f4400327882 (127.18.72.193:36185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4657edf9930f4029a47a7211c4beab3b: Network error: Client connection negotiation failed: client connection to 127.18.72.193:36185: connect: Connection refused (error 111)
I20260811 07:39:44.113127 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 30336
I20260811 07:39:44.123198 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 29946
2026-08-11T07:39:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2494 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:39:44 dist-test-slave-zkpd krb5kdc[30528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:39:44 dist-test-slave-zkpd krb5kdc[30528](info): set up 2 sockets
Aug 11 07:39:44 dist-test-slave-zkpd krb5kdc[30528](info): commencing operation
krb5kdc: starting...
W20260811 07:39:45.213503 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:39:45 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786433985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:39:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.72.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.72.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/rangeradmin_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/rangeradmin_127.18.72.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.72.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.72.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/rangerlookup_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/rangerlookup_127.18.72.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.72.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.72.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/HTTP_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/HTTP_127.18.72.212@KRBTEST.COM.keytab.
I20260811 07:39:45.378350 18723 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIvL2Zc/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/postgres -l logfile start
2026-08-11 07:39:47.740 UTC [30562] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 07:39:47.740 UTC [30562] LOG: listening on IPv4 address "127.18.72.212", port 38383
2026-08-11 07:39:47.743 UTC [30562] LOG: listening on Unix socket "/tmp/.s.PGSQL.38383"
2026-08-11 07:39:47.764 UTC [30565] LOG: database system was shut down at 2026-08-11 07:39:46 UTC
2026-08-11 07:39:47.767 UTC [30562] LOG: database system is ready to accept connections
I20260811 07:39:48.758018 18723 mini_postgres.cc:96] Postgres bound to 38383
2026-08-11 07:39:48.765 UTC [30574] FATAL: database "slave" does not exist
127.18.72.212:38383 - accepting connections
I20260811 07:39:48.766175 18723 mini_ranger.cc:161] Starting Ranger...
I20260811 07:39:48.781518 18723 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 07:39:49.207402 18723 mini_ranger.cc:87] Created ranger Postgres database
I20260811 07:39:49.207486 18723 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-admin
2026-08-11 07:39:49,503 [I] DB FLAVOR :POSTGRES
2026-08-11 07:39:49,504 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 07:39:49,504 [I] Checking connection..
2026-08-11 07:39:49,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 07:39:49,743 [I] Checking connection passed.
2026-08-11 07:39:49,744 [I] --------- Verifying version history table ---------
2026-08-11 07:39:49,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:39:49,992 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 07:39:49,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:39:50,249 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 07:39:50,250 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 07:39:50,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 07:39:50.500 UTC [30643] WARNING: there is no transaction in progress
2026-08-11 07:39:50,507 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 07:39:50,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:39:50,749 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 07:39:50,749 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 07:39:50,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 07:39:51,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 07:39:51,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zkpd.c.gcp-upstream.internal','N') ;"
2026-08-11 07:39:51,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 07:39:51,768 [I] Table x_portal_user does not exist in database ranger
2026-08-11 07:39:51,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 07:39:52,031 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 07:39:52,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 07:39:52,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 07:39:52,528 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 07:39:52,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 07:39:53.552 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.570 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.588 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.605 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.623 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.643 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.698 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.704 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.718 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.728 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.738 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.752 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.761 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.767 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.773 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:53.779 UTC [30762] WARNING: there is no transaction in progress
2026-08-11 07:39:54,364 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 07:39:54,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 07:39:54,606 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 07:39:54,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 07:39:54,869 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 07:39:54,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 07:39:55,119 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 07:39:55,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 07:39:55,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 07:39:55,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zkpd.c.gcp-upstream.internal';"
2026-08-11 07:39:55,854 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 07:39:55,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 07:39:56,097 [I] DB_PATCHES have already been applied
I20260811 07:39:56.104005 18723 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIvL2Zc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 07:39:56.104085 18723 mini_ranger.cc:193] Using host: 127.18.72.212
I20260811 07:39:56.108592 18723 mini_ranger.cc:227] Ranger admin URL: http://127.18.72.212:57069
Aug 11, 2026 7:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 7:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 7:39:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 7:39:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 7:39:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 07:39:57.273110 18723 mini_ranger.cc:160] Time spent starting Ranger: real 8.507s user 0.008s sys 0.004s
Aug 11, 2026 7:40:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:40:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 7:40:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:40:18 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:40:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 07:40:19.678013 18723 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.72.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.72.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/rangerkms_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/rangerkms_127.18.72.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.72.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.72.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.72.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/HTTP_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.72.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/HTTP_127.18.72.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 07:40:19 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434019, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 07:40:19.779065 18723 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 07:40:19.789906 18723 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 07:40:20.729559 18723 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 07:40:20.729668 18723 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms
I20260811 07:40:20.729779 18723 mini_ranger_kms.cc:227] Using postgres at 127.18.72.212:38383
2026-08-11 07:40:20,966 [I] DB FLAVOR :POSTGRES
2026-08-11 07:40:20,966 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 07:40:20,966 [I] Checking connection
2026-08-11 07:40:20,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 07:40:21,226 [I] connection success
2026-08-11 07:40:21,226 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 07:40:21,226 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 07:40:21,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 07:40:21,480 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 07:40:21,480 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 07:40:21,481 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 07:40:21,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:38383/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 07:40:21,735 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 07:40:22.216313 18723 mini_ranger_kms.cc:310] Created kms service
I20260811 07:40:22.426602 18723 mini_ranger_kms.cc:326] Created kudu user
I20260811 07:40:22.472486 18723 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 07:40:22 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434019, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:40:22.864725 18723 mini_ranger_kms.cc:384] Added ranger policy
I20260811 07:40:22.864895 18723 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 07:40:22.864923 18723 mini_ranger_kms.cc:240] Using host: 127.18.72.212
I20260811 07:40:22.869657 18723 mini_ranger_kms.cc:276] Ranger KMS PID: 30958
I20260811 07:40:22.869792 18723 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.72.212:50701
Aug 11, 2026 7:40:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:40:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:40:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:40:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:40:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:40:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 07:40:25.262383 18723 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.483s user 0.011s sys 0.207s
I20260811 07:40:25.262529 18723 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 07:40:25.262645 18723 mini_ranger_kms.cc:401] 127.18.72.212:50701/kms/v1/keys
Aug 11 07:40:25 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786434025, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.72.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 07:40:25 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434025, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.72.212@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
Aug 11 07:40:26 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434025, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.72.212@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
Aug 11 07:40:28 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:40:28.471575 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:33493
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:33493
--ranger_config_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:50701/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:40:28.619077 31019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:40:28.619324 31019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:40:28.619364 31019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:40:28.623391 31019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:40:28.623454 31019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:40:28.623474 31019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:40:28.623490 31019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:40:28.623505 31019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:40:28.629369 31019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.72.212:50701/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:33493
--ranger_config_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:33493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:40:28.630546 31019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:40:28.631704 31019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:40:28.640749 31026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:40:28.640987 31019 server_base.cc:1034] running on GCE node
W20260811 07:40:28.640765 31025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:40:28.640753 31028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:40:28.641841 31019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:40:28.642940 31019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:40:28.644133 31019 hybrid_clock.cc:648] HybridClock initialized: now 1786434028644116 us; error 33 us; skew 500 ppm
Aug 11 07:40:28 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434028, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:40:28.647250 31019 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:40:28.648206 31019 webserver.cc:460] Webserver started at http://127.18.72.254:33479/ using document root <none> and password file <none>
I20260811 07:40:28.648767 31019 fs_manager.cc:359] Metadata directory not provided
I20260811 07:40:28.648851 31019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:40:28.649083 31019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:40:28 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434028, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:40:28.702244 31019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "64d72c31507543efb35d466813b8ed94"
format_stamp: "Formatted at 2026-08-11 07:40:28 on dist-test-slave-zkpd"
server_key: "f14046a3b5b6e1a774c84c9150b4f375"
server_key_iv: "180599d52cc1552d296c5c2bbc2178e1"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:28.702885 31019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "64d72c31507543efb35d466813b8ed94"
format_stamp: "Formatted at 2026-08-11 07:40:28 on dist-test-slave-zkpd"
server_key: "f14046a3b5b6e1a774c84c9150b4f375"
server_key_iv: "180599d52cc1552d296c5c2bbc2178e1"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:28.707088 31019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:40:28.721148 31035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:40:28.722613 31019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:40:28.722764 31019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "64d72c31507543efb35d466813b8ed94"
format_stamp: "Formatted at 2026-08-11 07:40:28 on dist-test-slave-zkpd"
server_key: "f14046a3b5b6e1a774c84c9150b4f375"
server_key_iv: "180599d52cc1552d296c5c2bbc2178e1"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:28.722895 31019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:40:28.741732 31019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:40:28.744655 31019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:40:28.744858 31019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:40:28.753769 31019 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:33493
I20260811 07:40:28.753822 31088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:33493 every 8 connection(s)
I20260811 07:40:28.754769 31019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:40:28.757768 31089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:40:28.760607 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 31019
I20260811 07:40:28.760749 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:40:28.761051 18723 external_mini_cluster.cc:1428] Setting key db6a6c899f9ccb8d5ee266bb7a9ed95f
I20260811 07:40:28.764218 31089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94: Bootstrap starting.
I20260811 07:40:28.766743 31089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94: Neither blocks nor log segments found. Creating new log.
I20260811 07:40:28.767803 31089 log.cc:824] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94: Log is configured to *not* fsync() on all Append() calls
I20260811 07:40:28.770022 31089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94: No bootstrap required, opened a new log
Aug 11 07:40:28 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:40:28.774106 31089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d72c31507543efb35d466813b8ed94" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33493 } }
I20260811 07:40:28.774333 31089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:40:28.774407 31089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d72c31507543efb35d466813b8ed94, State: Initialized, Role: FOLLOWER
I20260811 07:40:28.774837 31089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d72c31507543efb35d466813b8ed94" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33493 } }
I20260811 07:40:28.774960 31089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:40:28.775084 31089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:40:28.775247 31089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:40:28.776391 31089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d72c31507543efb35d466813b8ed94" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33493 } }
I20260811 07:40:28.776768 31089 leader_election.cc:302] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64d72c31507543efb35d466813b8ed94; no voters:
I20260811 07:40:28.777101 31089 leader_election.cc:288] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:40:28.777213 31094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:40:28.777318 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:28.762569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.18.72.254:33493)
0811 07:40:28.763155 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:28.763204 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:28.763249 (+ 45us) server_negotiation.cc:407] Connection header received
0811 07:40:28.764153 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:28.764179 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:28.764607 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:28.764970 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:40:28.765845 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:28.767502 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:28.768637 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:28.769089 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:28.769287 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:28.772097 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:40:28.772103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:40:28.772116 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:40:28.772143 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:40:28.774018 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:28.774572 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:28.774584 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:28.774588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:28.774760 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:28.775061 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:28.775065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:28.775066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:28.775583 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:40:28.775868 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:28.776135 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 07:40:28.776418 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":130,"threads_started":1}
I20260811 07:40:28.777485 31094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [term 1 LEADER]: Becoming Leader. State: Replica: 64d72c31507543efb35d466813b8ed94, State: Running, Role: LEADER
I20260811 07:40:28.777812 31094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d72c31507543efb35d466813b8ed94" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33493 } }
I20260811 07:40:28.779477 31089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:40:28.779488 31095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64d72c31507543efb35d466813b8ed94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d72c31507543efb35d466813b8ed94" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33493 } } }
I20260811 07:40:28.779732 31095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [sys.catalog]: This master's current role is: LEADER
I20260811 07:40:28.780055 31094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64d72c31507543efb35d466813b8ed94. Latest consensus state: current_term: 1 leader_uuid: "64d72c31507543efb35d466813b8ed94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64d72c31507543efb35d466813b8ed94" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 33493 } } }
I20260811 07:40:28.780146 31094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94 [sys.catalog]: This master's current role is: LEADER
I20260811 07:40:28.780814 31098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:40:28.783367 31098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:40:28.783828 31089 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 07:40:28.794291 31098 catalog_manager.cc:1347] Generated new cluster ID: 766f2fc40e4a4a73ba03ac445ed79d49
I20260811 07:40:28.794379 31098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:40:28.809849 31098 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:40:28.810758 31098 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:40:28.818204 31098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94: Generated new TSK 0
I20260811 07:40:28.818851 31098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:40:29.575527 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:50701/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33493
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:40:29.752949 31135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:40:29.753288 31135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:40:29.753360 31135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:40:29.759091 31135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:40:29.759164 31135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:40:29.759246 31135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:40:29.766850 31135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.72.212:50701/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:40:29.768132 31135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:40:29.769186 31135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:40:29.782075 31140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:40:29.782075 31141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:40:29.782473 31143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:40:29.782804 31135 server_base.cc:1034] running on GCE node
I20260811 07:40:29.783346 31135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:40:29.784241 31135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:40:29.785418 31135 hybrid_clock.cc:648] HybridClock initialized: now 1786434029785378 us; error 57 us; skew 500 ppm
Aug 11 07:40:29 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:40:29.789217 31135 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:40:29.790227 31135 webserver.cc:460] Webserver started at http://127.18.72.193:41819/ using document root <none> and password file <none>
I20260811 07:40:29.790819 31135 fs_manager.cc:359] Metadata directory not provided
I20260811 07:40:29.790912 31135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:40:29.791149 31135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:40:29 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:40:29.807034 31135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77"
format_stamp: "Formatted at 2026-08-11 07:40:29 on dist-test-slave-zkpd"
server_key: "75542364e21cde35c78e9fb7ec20b5ff"
server_key_iv: "482d4c26d5f7f14223f1b6f9d01ebd4b"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:29.807818 31135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77"
format_stamp: "Formatted at 2026-08-11 07:40:29 on dist-test-slave-zkpd"
server_key: "75542364e21cde35c78e9fb7ec20b5ff"
server_key_iv: "482d4c26d5f7f14223f1b6f9d01ebd4b"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:29.812687 31135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 07:40:29.827533 31155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:40:29.829217 31135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:40:29.829346 31135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77"
format_stamp: "Formatted at 2026-08-11 07:40:29 on dist-test-slave-zkpd"
server_key: "75542364e21cde35c78e9fb7ec20b5ff"
server_key_iv: "482d4c26d5f7f14223f1b6f9d01ebd4b"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:29.829468 31135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:40:29.864950 31135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:40:29.868450 31135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:40:29.868696 31135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:40:29.869401 31135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:40:29.870478 31135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:40:29.870582 31135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:40:29.870705 31135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:40:29.870810 31135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:40:29.892333 31135 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:39787
I20260811 07:40:29.893678 31135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:40:29.893867 31274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:39787 every 8 connection(s)
I20260811 07:40:29.898904 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 31135
I20260811 07:40:29.899044 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:40:29.899362 18723 external_mini_cluster.cc:1428] Setting key 5f7e094ec836f41feda4b59dc60a9fd5
Aug 11 07:40:29 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:40:29.914851 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:29.896676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33493 (local address 127.18.72.193:50643)
0811 07:40:29.897532 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:29.897600 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 07:40:29.898927 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:40:29.900179 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:40:29.900190 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:40:29.900603 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:40:29.901247 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:29.901272 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:29.902553 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:29.902557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:29.904017 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:29.904029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:29.905607 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:29.905612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:29.905827 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:29.906550 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:40:29.906572 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:40:29.908576 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:40:29.910998 (+ 2422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:40:29.911004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:40:29.911021 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:40:29.911445 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:40:29.911806 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:40:29.911810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:40:29.911812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:40:29.912025 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:40:29.912544 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:40:29.912553 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:40:29.912924 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 07:40:29.913156 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 07:40:29.913525 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":154,"threads_started":1}
I20260811 07:40:29.915637 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:29.896901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:50643 (local address 127.18.72.254:33493)
0811 07:40:29.899533 (+ 2632us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:29.899538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:29.899556 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:40:29.899606 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:29.899609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:29.899664 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:29.900611 (+ 947us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:40:29.901397 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:29.902410 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:29.904934 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:29.905312 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:29.905421 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:29.908807 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:40:29.908811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:40:29.908814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:40:29.908841 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:40:29.910862 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:29.911567 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:29.911570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:29.911572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:29.911663 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:29.912145 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:29.912148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:29.912150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:29.912381 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:40:29.912486 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:29.915321 (+ 2835us) server_negotiation.cc:299] Negotiation successful
0811 07:40:29.915463 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2530,"thread_start_us":82,"threads_started":1}
I20260811 07:40:29.917457 31275 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33493
I20260811 07:40:29.917729 31275 heartbeater.cc:464] Registering TS with master...
I20260811 07:40:29.918370 31275 heartbeater.cc:511] Master 127.18.72.254:33493 requested a full tablet report, sending...
I20260811 07:40:29.920183 31052 ts_manager.cc:194] Registered new tserver with Master: 1f0ecfee0d7a4337a1dc5e2dfe274e77 (127.18.72.193:39787)
I20260811 07:40:29.921777 31052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:50643
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Aug 11 07:40:30 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
W20260811 07:40:30.354696 18723 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.72.194: real 0.452s user 0.000s sys 0.004s
I20260811 07:40:30.355427 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:50701/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33493
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:40:30.520826 31285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:40:30.521188 31285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:40:30.521256 31285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:40:30.527462 31285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:40:30.527597 31285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:40:30.527760 31285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:40:30.535806 31285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.72.212:50701/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:40:30.537439 31285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:40:30.538947 31285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:40:30.552138 31295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:40:30.552280 31292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:40:30.552682 31293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:40:30.552851 31285 server_base.cc:1034] running on GCE node
I20260811 07:40:30.553395 31285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:40:30.554226 31285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:40:30.555434 31285 hybrid_clock.cc:648] HybridClock initialized: now 1786434030555409 us; error 38 us; skew 500 ppm
Aug 11 07:40:30 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:40:30.559522 31285 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:40:30.560832 31285 webserver.cc:460] Webserver started at http://127.18.72.194:42777/ using document root <none> and password file <none>
I20260811 07:40:30.561754 31285 fs_manager.cc:359] Metadata directory not provided
I20260811 07:40:30.561874 31285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:40:30.562155 31285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:40:30 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:40:30.580147 31285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "a5878ef4db824b8b8bd35dafef59d6ea"
format_stamp: "Formatted at 2026-08-11 07:40:30 on dist-test-slave-zkpd"
server_key: "bc704617180e925b0984e51d447697c9"
server_key_iv: "a04b45dd45b7c5fe2011365af71769e8"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:30.581261 31285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "a5878ef4db824b8b8bd35dafef59d6ea"
format_stamp: "Formatted at 2026-08-11 07:40:30 on dist-test-slave-zkpd"
server_key: "bc704617180e925b0984e51d447697c9"
server_key_iv: "a04b45dd45b7c5fe2011365af71769e8"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:30.589481 31285 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.004s
I20260811 07:40:30.608412 31302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:40:30.610252 31285 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260811 07:40:30.610395 31285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "a5878ef4db824b8b8bd35dafef59d6ea"
format_stamp: "Formatted at 2026-08-11 07:40:30 on dist-test-slave-zkpd"
server_key: "bc704617180e925b0984e51d447697c9"
server_key_iv: "a04b45dd45b7c5fe2011365af71769e8"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:30.610531 31285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:40:30.643620 31285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:40:30.647208 31285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:40:30.647459 31285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:40:30.648204 31285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:40:30.649263 31285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:40:30.649335 31285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:40:30.649430 31285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:40:30.649693 31285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 07:40:30 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:40:30.664624 31285 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:36907
I20260811 07:40:30.665154 31422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:36907 every 8 connection(s)
I20260811 07:40:30.665930 31285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:40:30.674329 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 31285
I20260811 07:40:30.674468 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:40:30.674780 18723 external_mini_cluster.cc:1428] Setting key 965a6c3d3224b87123aecf376e5cbde3
Aug 11 07:40:30 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:40:30.693766 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:30.669792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:34559 (local address 127.18.72.254:33493)
0811 07:40:30.670037 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:30.670042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:30.670955 (+ 913us) server_negotiation.cc:407] Connection header received
0811 07:40:30.672370 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:30.672373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:30.672433 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:30.672524 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:40:30.674317 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:30.675133 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:30.677340 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:30.678221 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:30.678497 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:30.687145 (+ 8648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:40:30.687150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:40:30.687154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:40:30.687188 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:40:30.689295 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:30.690035 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:30.690038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:30.690040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:30.690141 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:30.692325 (+ 2184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:30.692329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:30.692330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:30.692576 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:40:30.692687 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:30.693482 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 07:40:30.693609 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":81,"threads_started":1}
I20260811 07:40:30.696935 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:30.668664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33493 (local address 127.18.72.194:34559)
0811 07:40:30.670778 (+ 2114us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:30.670835 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 07:40:30.671987 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:40:30.672897 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:40:30.672908 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:40:30.673302 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:40:30.674115 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:30.674140 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:30.675300 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:30.675303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:30.676668 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:30.676680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:30.678705 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:30.678710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:30.680560 (+ 1850us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:30.683478 (+ 2918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:40:30.683501 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:40:30.686522 (+ 3021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:40:30.689414 (+ 2892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:40:30.689420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:40:30.689438 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:40:30.689804 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:40:30.691988 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:40:30.691991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:40:30.691993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:40:30.692202 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:40:30.692757 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:40:30.692764 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:40:30.693145 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 07:40:30.693984 (+ 839us) client_negotiation.cc:240] Negotiation successful
0811 07:40:30.694368 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1851,"thread_start_us":127,"threads_started":1}
I20260811 07:40:30.698230 31423 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33493
I20260811 07:40:30.698474 31423 heartbeater.cc:464] Registering TS with master...
I20260811 07:40:30.699075 31423 heartbeater.cc:511] Master 127.18.72.254:33493 requested a full tablet report, sending...
I20260811 07:40:30.700392 31052 ts_manager.cc:194] Registered new tserver with Master: a5878ef4db824b8b8bd35dafef59d6ea (127.18.72.194:36907)
I20260811 07:40:30.701139 31052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:34559
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:40:30.730809 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:50701/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33493
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:33805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:40:30.905104 31430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:40:30.905496 31430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:40:30.905589 31430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:40:30.911429 31430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:40:30.911504 31430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:40:30.911584 31430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:40:30.919382 31430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:33805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.72.212:50701/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:33493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:40:30.920765 31430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:40:30.922153 31430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:40:30.924844 31275 heartbeater.cc:503] Master 127.18.72.254:33493 was elected leader, sending a full tablet report...
W20260811 07:40:30.935240 31436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:40:30.935877 31439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:40:30.936590 31437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:40:30.937168 31430 server_base.cc:1034] running on GCE node
I20260811 07:40:30.937673 31430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:40:30.938421 31430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:40:30.939661 31430 hybrid_clock.cc:648] HybridClock initialized: now 1786434030939635 us; error 37 us; skew 500 ppm
Aug 11 07:40:30 dist-test-slave-zkpd krb5kdc[30528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:40:30.943611 31430 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:40:30.944795 31430 webserver.cc:460] Webserver started at http://127.18.72.195:37481/ using document root <none> and password file <none>
I20260811 07:40:30.945571 31430 fs_manager.cc:359] Metadata directory not provided
I20260811 07:40:30.945667 31430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:40:30.946003 31430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:40:30 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:40:30.965813 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f"
format_stamp: "Formatted at 2026-08-11 07:40:30 on dist-test-slave-zkpd"
server_key: "6330f7021b1e069b586a2c93b576ec58"
server_key_iv: "fe1faf069ddec5b6f202d62358e9b812"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:30.966760 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f"
format_stamp: "Formatted at 2026-08-11 07:40:30 on dist-test-slave-zkpd"
server_key: "6330f7021b1e069b586a2c93b576ec58"
server_key_iv: "fe1faf069ddec5b6f202d62358e9b812"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:30.977451 31430 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260811 07:40:31.000568 31448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Aug 11 07:40:30 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:40:31.005548 31430 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.001s sys 0.000s
I20260811 07:40:31.005699 31430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f"
format_stamp: "Formatted at 2026-08-11 07:40:30 on dist-test-slave-zkpd"
server_key: "6330f7021b1e069b586a2c93b576ec58"
server_key_iv: "fe1faf069ddec5b6f202d62358e9b812"
server_key_version: "kuduclusterkey@0"
I20260811 07:40:31.005867 31430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:40:31.075153 31430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:40:31.079157 31430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:40:31.079442 31430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:40:31.080380 31430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:40:31.081769 31430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:40:31.081961 31430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:40:31.082067 31430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:40:31.082137 31430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:40:31.098548 31430 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:43341
I20260811 07:40:31.099746 31430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:40:31.101737 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 31430
I20260811 07:40:31.101877 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:40:31.102206 18723 external_mini_cluster.cc:1428] Setting key 491add2831342cb1724006b99f5cc672
I20260811 07:40:31.113509 31577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:43341 every 8 connection(s)
Aug 11 07:40:31 dist-test-slave-zkpd krb5kdc[30528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:40:31.138262 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.116846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:59355 (local address 127.18.72.254:33493)
0811 07:40:31.117579 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:31.117585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:31.123537 (+ 5952us) server_negotiation.cc:407] Connection header received
0811 07:40:31.124972 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:31.124976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:31.125033 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:31.125088 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:40:31.126850 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.127701 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.129471 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.129807 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.129896 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.133137 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:40:31.133141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:40:31.133144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:40:31.133170 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:40:31.135417 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:31.136213 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:31.136217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:31.136219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:31.136320 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:31.136750 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:31.136753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:31.136755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:31.137029 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:40:31.137123 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:31.137906 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 07:40:31.138065 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627}
I20260811 07:40:31.139232 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.116483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:33493 (local address 127.18.72.195:59355)
0811 07:40:31.123367 (+ 6884us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:31.123431 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 07:40:31.124692 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:40:31.125533 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:40:31.125542 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:40:31.125974 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:40:31.126655 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.126687 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.127885 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.127929 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.129334 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.129345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.130135 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.130140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.130342 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.131076 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:40:31.131098 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:40:31.132945 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:40:31.135640 (+ 2695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:40:31.135647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:40:31.135662 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:40:31.136087 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:40:31.136437 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:40:31.136440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:40:31.136442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:40:31.136616 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:40:31.137197 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:40:31.137203 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:40:31.137537 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 07:40:31.137813 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 07:40:31.138156 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6617,"thread_start_us":167,"threads_started":1}
I20260811 07:40:31.140602 31578 heartbeater.cc:347] Connected to a master server at 127.18.72.254:33493
I20260811 07:40:31.140913 31578 heartbeater.cc:464] Registering TS with master...
I20260811 07:40:31.141552 31578 heartbeater.cc:511] Master 127.18.72.254:33493 requested a full tablet report, sending...
I20260811 07:40:31.142755 31052 ts_manager.cc:194] Registered new tserver with Master: ad8a31ee754b4c43aaf38cd2f22b9b1f (127.18.72.195:43341)
I20260811 07:40:31.143471 31052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:59355
I20260811 07:40:31.153862 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:40:31.166435 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.156276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.18.72.254:33493)
0811 07:40:31.156406 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:31.156410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:31.156690 (+ 280us) server_negotiation.cc:407] Connection header received
0811 07:40:31.156813 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:31.156816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:31.156866 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:31.156946 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:40:31.157970 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.158791 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.160105 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.160470 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.160556 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.162550 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:40:31.162554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:40:31.162556 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:40:31.162585 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:40:31.164499 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:31.165007 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:31.165011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:31.165012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:31.165101 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:40:31.165469 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:40:31.165473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:40:31.165475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:40:31.165704 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:40:31.165858 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:31.166129 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 07:40:31.166268 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:40:31.169821 31052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:40:31.174598 31052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:40:31.194355 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.186248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39787 (local address 127.0.0.1:44792)
0811 07:40:31.188850 (+ 2602us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:31.188863 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:40:31.188965 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:40:31.189481 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:40:31.189488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:40:31.189523 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:40:31.189666 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.189674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.191241 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.191245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.192613 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.192624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.193825 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.193829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.193937 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.193982 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 07:40:31.194074 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 07:40:31.194201 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2534,"thread_start_us":65,"threads_started":1}
I20260811 07:40:31.194651 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.186129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:36907 (local address 127.0.0.1:36740)
0811 07:40:31.188377 (+ 2248us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:31.188428 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 07:40:31.188566 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:40:31.190506 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:40:31.190510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:40:31.190530 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:40:31.190625 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.190632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.192080 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.192084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.193155 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.193164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.194267 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.194270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.194381 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.194415 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 07:40:31.194474 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 07:40:31.194543 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2059,"thread_start_us":95,"threads_started":1}
I20260811 07:40:31.195623 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.186981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36740 (local address 127.18.72.194:36907)
0811 07:40:31.189218 (+ 2237us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:31.189236 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:31.190027 (+ 791us) server_negotiation.cc:407] Connection header received
0811 07:40:31.190111 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:31.190117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:31.190324 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:31.190520 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:40:31.190786 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.191951 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.193335 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.194140 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.195107 (+ 967us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.195325 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:31.195374 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 07:40:31.195472 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2118,"thread_start_us":92,"threads_started":1}
I20260811 07:40:31.196204 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.187120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.18.72.193:39787)
0811 07:40:31.188111 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:31.188121 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:31.188979 (+ 858us) server_negotiation.cc:407] Connection header received
0811 07:40:31.189071 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:31.189077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:31.189291 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:31.189851 (+ 560us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:40:31.189996 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.191110 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.192765 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.193657 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.195174 (+ 1517us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.195315 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:31.195374 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 07:40:31.196080 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":73,"threads_started":1}
I20260811 07:40:31.199327 31201 tablet_service.cc:1467] Processing CreateTablet for tablet ee55224422bd444d880a7d2160dc1a60 (DEFAULT_TABLE table=test-table [id=8fb575c8413d4f5eab33c14cad7468e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:40:31.199375 31351 tablet_service.cc:1467] Processing CreateTablet for tablet ee55224422bd444d880a7d2160dc1a60 (DEFAULT_TABLE table=test-table [id=8fb575c8413d4f5eab33c14cad7468e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:40:31.200589 31201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee55224422bd444d880a7d2160dc1a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:40:31.200589 31351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee55224422bd444d880a7d2160dc1a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:40:31.212018 31599 tablet_bootstrap.cc:492] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Bootstrap starting.
I20260811 07:40:31.214681 31599 tablet_bootstrap.cc:654] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Neither blocks nor log segments found. Creating new log.
I20260811 07:40:31.215364 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.187885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:43341 (local address 127.0.0.1:55404)
0811 07:40:31.194791 (+ 6906us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:31.194804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:40:31.194878 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:40:31.202503 (+ 7625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:40:31.202506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:40:31.202528 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:40:31.202628 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.202635 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.211626 (+ 8991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.211633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.212742 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.212754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.214128 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.214132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.214219 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.215067 (+ 848us) client_negotiation.cc:769] Sending connection context
0811 07:40:31.215142 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 07:40:31.215223 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6839,"thread_start_us":87,"threads_started":1}
I20260811 07:40:31.215996 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.188173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.18.72.195:43341)
0811 07:40:31.201958 (+ 13785us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:31.201970 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:31.201996 (+ 26us) server_negotiation.cc:407] Connection header received
0811 07:40:31.202088 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:31.202094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:31.202300 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:31.202528 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:40:31.210655 (+ 8127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.211490 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.212943 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.213794 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.213913 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.214062 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:31.215658 (+ 1596us) server_negotiation.cc:299] Negotiation successful
0811 07:40:31.215806 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13657,"thread_start_us":102,"threads_started":1}
I20260811 07:40:31.216035 31599 log.cc:824] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Log is configured to *not* fsync() on all Append() calls
I20260811 07:40:31.216800 31600 tablet_bootstrap.cc:492] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77: Bootstrap starting.
I20260811 07:40:31.218853 31600 tablet_bootstrap.cc:654] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77: Neither blocks nor log segments found. Creating new log.
I20260811 07:40:31.219527 31600 log.cc:824] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77: Log is configured to *not* fsync() on all Append() calls
I20260811 07:40:31.219273 31498 tablet_service.cc:1467] Processing CreateTablet for tablet ee55224422bd444d880a7d2160dc1a60 (DEFAULT_TABLE table=test-table [id=8fb575c8413d4f5eab33c14cad7468e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:40:31.220506 31498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee55224422bd444d880a7d2160dc1a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:40:31.222123 31600 tablet_bootstrap.cc:492] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77: No bootstrap required, opened a new log
I20260811 07:40:31.222342 31600 ts_tablet_manager.cc:1397] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 07:40:31.225373 31599 tablet_bootstrap.cc:492] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: No bootstrap required, opened a new log
I20260811 07:40:31.225733 31599 ts_tablet_manager.cc:1397] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260811 07:40:31.226500 31600 raft_consensus.cc:348] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.226863 31600 raft_consensus.cc:374] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:40:31.226955 31600 raft_consensus.cc:729] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0ecfee0d7a4337a1dc5e2dfe274e77, State: Initialized, Role: FOLLOWER
I20260811 07:40:31.227496 31600 consensus_queue.cc:260] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.228711 31600 ts_tablet_manager.cc:1428] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77: Time spent starting tablet: real 0.006s user 0.002s sys 0.006s
I20260811 07:40:31.230521 31604 tablet_bootstrap.cc:492] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Bootstrap starting.
I20260811 07:40:31.231241 31599 raft_consensus.cc:348] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.231571 31599 raft_consensus.cc:374] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:40:31.231652 31599 raft_consensus.cc:729] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5878ef4db824b8b8bd35dafef59d6ea, State: Initialized, Role: FOLLOWER
I20260811 07:40:31.232153 31599 consensus_queue.cc:260] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.233289 31599 ts_tablet_manager.cc:1428] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 07:40:31.233415 31604 tablet_bootstrap.cc:654] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Neither blocks nor log segments found. Creating new log.
I20260811 07:40:31.233891 31423 heartbeater.cc:503] Master 127.18.72.254:33493 was elected leader, sending a full tablet report...
I20260811 07:40:31.234357 31604 log.cc:824] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:40:31.236598 31604 tablet_bootstrap.cc:492] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: No bootstrap required, opened a new log
I20260811 07:40:31.236825 31604 ts_tablet_manager.cc:1397] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260811 07:40:31.241070 31604 raft_consensus.cc:348] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.241411 31604 raft_consensus.cc:374] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:40:31.241510 31604 raft_consensus.cc:729] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8a31ee754b4c43aaf38cd2f22b9b1f, State: Initialized, Role: FOLLOWER
I20260811 07:40:31.242025 31604 consensus_queue.cc:260] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.243036 31604 ts_tablet_manager.cc:1428] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260811 07:40:31.243422 31578 heartbeater.cc:503] Master 127.18.72.254:33493 was elected leader, sending a full tablet report...
I20260811 07:40:31.359405 31603 raft_consensus.cc:482] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:40:31.359767 31603 raft_consensus.cc:504] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.361418 31603 leader_election.cc:288] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5878ef4db824b8b8bd35dafef59d6ea (127.18.72.194:36907), ad8a31ee754b4c43aaf38cd2f22b9b1f (127.18.72.195:43341)
W20260811 07:40:31.365829 31579 tablet.cc:2369] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:40:31.367344 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.362262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:58579 (local address 127.18.72.194:36907)
0811 07:40:31.362410 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:31.362416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:31.362434 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:40:31.362621 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:31.362624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:31.362683 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:31.362771 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:40:31.363421 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.364015 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.365743 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.366481 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.366577 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.366641 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:31.367082 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 07:40:31.367187 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:40:31.367378 31617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.361746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:36907 (local address 127.18.72.193:58579)
0811 07:40:31.362326 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:31.362343 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:40:31.362473 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:40:31.362947 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:40:31.362949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:40:31.362976 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:40:31.363232 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.363243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.364142 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.364146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.365587 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.365594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.366726 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.366729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.366938 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.366956 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 07:40:31.367028 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:40:31.367156 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":124,"thread_start_us":217,"threads_started":1}
I20260811 07:40:31.367479 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.361881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:48835 (local address 127.18.72.195:43341)
0811 07:40:31.362072 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:31.362078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:31.362635 (+ 557us) server_negotiation.cc:407] Connection header received
0811 07:40:31.362751 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:31.362754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:31.362813 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:31.362894 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:40:31.363387 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.364187 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.365743 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.366389 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.366503 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.366575 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:31.367251 (+ 676us) server_negotiation.cc:299] Negotiation successful
0811 07:40:31.367347 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 07:40:31.367803 31618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.361744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:43341 (local address 127.18.72.193:48835)
0811 07:40:31.362472 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:31.362493 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 07:40:31.362641 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:40:31.362937 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:40:31.362940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:40:31.363023 (+ 83us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:40:31.363256 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.363262 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.364407 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.364411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.365580 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:40:31.365594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.366705 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.366710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:40:31.366883 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.366940 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 07:40:31.367028 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 07:40:31.367152 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":211,"spinlock_wait_cycles":26368,"thread_start_us":193,"threads_started":1}
I20260811 07:40:31.368373 31354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee55224422bd444d880a7d2160dc1a60" candidate_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" is_pre_election: true
I20260811 07:40:31.368558 31527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee55224422bd444d880a7d2160dc1a60" candidate_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" is_pre_election: true
I20260811 07:40:31.368818 31527 raft_consensus.cc:2454] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0ecfee0d7a4337a1dc5e2dfe274e77 in term 0.
I20260811 07:40:31.368819 31354 raft_consensus.cc:2454] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0ecfee0d7a4337a1dc5e2dfe274e77 in term 0.
I20260811 07:40:31.369494 31159 leader_election.cc:302] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f0ecfee0d7a4337a1dc5e2dfe274e77, a5878ef4db824b8b8bd35dafef59d6ea, ad8a31ee754b4c43aaf38cd2f22b9b1f; no voters:
I20260811 07:40:31.369902 31603 raft_consensus.cc:2781] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:40:31.370042 31603 raft_consensus.cc:482] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:40:31.370095 31603 raft_consensus.cc:3037] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:40:31.372025 31603 raft_consensus.cc:504] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.372568 31603 leader_election.cc:288] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [CANDIDATE]: Term 1 election: Requested vote from peers a5878ef4db824b8b8bd35dafef59d6ea (127.18.72.194:36907), ad8a31ee754b4c43aaf38cd2f22b9b1f (127.18.72.195:43341)
I20260811 07:40:31.373265 31354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee55224422bd444d880a7d2160dc1a60" candidate_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5878ef4db824b8b8bd35dafef59d6ea"
I20260811 07:40:31.373286 31527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee55224422bd444d880a7d2160dc1a60" candidate_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f"
I20260811 07:40:31.373452 31527 raft_consensus.cc:3037] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:40:31.373473 31354 raft_consensus.cc:3037] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:40:31.374816 31354 raft_consensus.cc:2454] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0ecfee0d7a4337a1dc5e2dfe274e77 in term 1.
I20260811 07:40:31.375034 31527 raft_consensus.cc:2454] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0ecfee0d7a4337a1dc5e2dfe274e77 in term 1.
I20260811 07:40:31.375324 31159 leader_election.cc:302] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0ecfee0d7a4337a1dc5e2dfe274e77, a5878ef4db824b8b8bd35dafef59d6ea; no voters:
I20260811 07:40:31.375555 31603 raft_consensus.cc:2781] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:40:31.375887 31603 raft_consensus.cc:686] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [term 1 LEADER]: Becoming Leader. State: Replica: 1f0ecfee0d7a4337a1dc5e2dfe274e77, State: Running, Role: LEADER
I20260811 07:40:31.376333 31603 consensus_queue.cc:237] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } }
I20260811 07:40:31.380749 31052 catalog_manager.cc:5495] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f0ecfee0d7a4337a1dc5e2dfe274e77 (127.18.72.193). New cstate: current_term: 1 leader_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f0ecfee0d7a4337a1dc5e2dfe274e77" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 } health_report { overall_health: UNKNOWN } } }
W20260811 07:40:31.396409 31276 tablet.cc:2369] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:40:31.427907 31424 tablet.cc:2369] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:40:31.437346 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.432639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44804 (local address 127.18.72.193:39787)
0811 07:40:31.432789 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:40:31.432793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:40:31.432811 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:40:31.432971 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:40:31.432974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:40:31.433050 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:40:31.433179 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:40:31.433520 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.434432 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.435540 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:40:31.435942 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:40:31.436072 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:40:31.436436 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:40:31.436908 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:40:31.437057 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 07:40:31.437095 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 07:40:31.437158 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 07:40:31.444608 31527 raft_consensus.cc:1261] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 1 FOLLOWER]: Refusing update from remote peer 1f0ecfee0d7a4337a1dc5e2dfe274e77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:40:31.444644 31354 raft_consensus.cc:1261] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 1 FOLLOWER]: Refusing update from remote peer 1f0ecfee0d7a4337a1dc5e2dfe274e77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:40:31.445612 31620 consensus_queue.cc:1035] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5878ef4db824b8b8bd35dafef59d6ea" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 36907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:40:31.445808 31603 consensus_queue.cc:1035] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad8a31ee754b4c43aaf38cd2f22b9b1f" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 43341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:40:31.454172 31623 mvcc.cc:205] Tried to move back new op lower bound from 7317233792789213184 to 7317233792521015296. Current Snapshot: MvccSnapshot[applied={T|T < 7317233792789213184}]
I20260811 07:40:31.461300 31625 mvcc.cc:205] Tried to move back new op lower bound from 7317233792789213184 to 7317233792521015296. Current Snapshot: MvccSnapshot[applied={T|T < 7317233792789213184}]
I20260811 07:40:31.468333 31052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:40:31.468523 31052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:40:31.472987 31052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64d72c31507543efb35d466813b8ed94: Sending DeleteTablet for 3 replicas of tablet ee55224422bd444d880a7d2160dc1a60
I20260811 07:40:31.474215 31498 tablet_service.cc:1514] Processing DeleteTablet for tablet ee55224422bd444d880a7d2160dc1a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:40:31 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:55404
I20260811 07:40:31.474821 31632 tablet_replica.cc:330] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: stopping tablet replica
I20260811 07:40:31.474782 31351 tablet_service.cc:1514] Processing DeleteTablet for tablet ee55224422bd444d880a7d2160dc1a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:40:31 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:36740
I20260811 07:40:31.475155 31632 raft_consensus.cc:2229] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:40:31.475535 31632 raft_consensus.cc:2258] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:40:31.476136 31201 tablet_service.cc:1514] Processing DeleteTablet for tablet ee55224422bd444d880a7d2160dc1a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:40:31 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:44792
I20260811 07:40:31.476668 31634 tablet_replica.cc:330] T ee55224422bd444d880a7d2160dc1a60 P 1f0ecfee0d7a4337a1dc5e2dfe274e77: stopping tablet replica
I20260811 07:40:31.478180 31632 ts_tablet_manager.cc:1905] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:40:31.478309 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 31135
I20260811 07:40:31.482417 31633 tablet_replica.cc:330] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: stopping tablet replica
I20260811 07:40:31.482882 31633 raft_consensus.cc:2229] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:40:31.483289 31633 raft_consensus.cc:2258] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:40:31.484624 31632 ts_tablet_manager.cc:1918] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:40:31.484758 31632 log.cc:1196] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/ee55224422bd444d880a7d2160dc1a60
I20260811 07:40:31.485134 31632 ts_tablet_manager.cc:1939] T ee55224422bd444d880a7d2160dc1a60 P ad8a31ee754b4c43aaf38cd2f22b9b1f: Deleting consensus metadata
I20260811 07:40:31.485426 31633 ts_tablet_manager.cc:1905] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:40:31.486449 31037 catalog_manager.cc:4841] TS ad8a31ee754b4c43aaf38cd2f22b9b1f (127.18.72.195:43341): tablet ee55224422bd444d880a7d2160dc1a60 (table test-table [id=8fb575c8413d4f5eab33c14cad7468e6]) successfully deleted
I20260811 07:40:31.489154 31633 ts_tablet_manager.cc:1918] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:40:31.489310 31633 log.cc:1196] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/ee55224422bd444d880a7d2160dc1a60
I20260811 07:40:31.489686 31633 ts_tablet_manager.cc:1939] T ee55224422bd444d880a7d2160dc1a60 P a5878ef4db824b8b8bd35dafef59d6ea: Deleting consensus metadata
I20260811 07:40:31.491240 31039 catalog_manager.cc:4841] TS a5878ef4db824b8b8bd35dafef59d6ea (127.18.72.194:36907): tablet ee55224422bd444d880a7d2160dc1a60 (table test-table [id=8fb575c8413d4f5eab33c14cad7468e6]) successfully deleted
W20260811 07:40:31.492040 31038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:39787 (error 108)
I20260811 07:40:31.492916 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 31285
I20260811 07:40:31.493672 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:40:31.492870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39787 (local address 127.0.0.1:44806)
0811 07:40:31.493501 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 07:40:31.493587 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541}
W20260811 07:40:31.495240 31038 catalog_manager.cc:4584] TS 1f0ecfee0d7a4337a1dc5e2dfe274e77 (127.18.72.193:39787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee55224422bd444d880a7d2160dc1a60: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39787: connect: Connection refused (error 111)
I20260811 07:40:31.502031 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 31430
I20260811 07:40:31.509660 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 31019
I20260811 07:40:31.521092 18723 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 07:40:31.866133 18723 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 07:40:31.866315 18723 mini_ranger.cc:66] Stopping Ranger...
I20260811 07:40:32.210990 18723 mini_ranger.cc:68] Stopped Ranger
2026-08-11 07:40:32.211 UTC [30562] LOG: received smart shutdown request
2026-08-11 07:40:32.215 UTC [30562] LOG: background worker "logical replication launcher" (PID 30572) exited with exit code 1
2026-08-11 07:40:32.215 UTC [30566] LOG: shutting down
2026-08-11 07:40:32.354 UTC [30562] LOG: database system is shut down
2026-08-11T07:40:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48358 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:40:32 dist-test-slave-zkpd krb5kdc[31645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:40:32 dist-test-slave-zkpd krb5kdc[31645](info): set up 2 sockets
Aug 11 07:40:32 dist-test-slave-zkpd krb5kdc[31645](info): commencing operation
krb5kdc: starting...
W20260811 07:40:33.575443 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:40:33 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:40:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:40:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.72.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.72.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/rangeradmin_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/rangeradmin_127.18.72.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.72.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.72.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/rangerlookup_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/rangerlookup_127.18.72.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.72.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.72.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/HTTP_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/HTTP_127.18.72.212@KRBTEST.COM.keytab.
I20260811 07:40:33.738662 18723 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskIvL2Zc/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/postgres -l logfile start
2026-08-11 07:40:36.134 UTC [31677] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 07:40:36.134 UTC [31677] LOG: listening on IPv4 address "127.18.72.212", port 54171
2026-08-11 07:40:36.137 UTC [31677] LOG: listening on Unix socket "/tmp/.s.PGSQL.54171"
2026-08-11 07:40:36.159 UTC [31680] LOG: database system was shut down at 2026-08-11 07:40:34 UTC
2026-08-11 07:40:36.163 UTC [31677] LOG: database system is ready to accept connections
I20260811 07:40:37.149745 18723 mini_postgres.cc:96] Postgres bound to 54171
2026-08-11 07:40:37.157 UTC [31688] FATAL: database "slave" does not exist
127.18.72.212:54171 - accepting connections
I20260811 07:40:37.157982 18723 mini_ranger.cc:161] Starting Ranger...
I20260811 07:40:37.173928 18723 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 07:40:37.611073 18723 mini_ranger.cc:87] Created ranger Postgres database
I20260811 07:40:37.611148 18723 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-admin
2026-08-11 07:40:37,896 [I] DB FLAVOR :POSTGRES
2026-08-11 07:40:37,896 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 07:40:37,896 [I] Checking connection..
2026-08-11 07:40:37,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 07:40:38,141 [I] Checking connection passed.
2026-08-11 07:40:38,141 [I] --------- Verifying version history table ---------
2026-08-11 07:40:38,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:40:38,407 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 07:40:38,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:40:38,667 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 07:40:38,668 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 07:40:38,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 07:40:38.922 UTC [31755] WARNING: there is no transaction in progress
2026-08-11 07:40:38,929 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 07:40:38,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 07:40:39,190 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 07:40:39,191 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 07:40:39,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 07:40:39,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 07:40:39,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zkpd.c.gcp-upstream.internal','N') ;"
2026-08-11 07:40:39,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 07:40:40,195 [I] Table x_portal_user does not exist in database ranger
2026-08-11 07:40:40,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 07:40:40,446 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 07:40:40,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 07:40:40,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 07:40:40,958 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 07:40:40,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 07:40:41.855 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:41.870 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:41.886 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:41.900 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:41.915 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:41.931 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:41.986 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:41.992 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42.005 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42.015 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42.024 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42.036 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42.046 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42.053 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42.059 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42.066 UTC [31868] WARNING: there is no transaction in progress
2026-08-11 07:40:42,658 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 07:40:42,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 07:40:42,886 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 07:40:42,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 07:40:43,151 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 07:40:43,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 07:40:43,397 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 07:40:43,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 07:40:43,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 07:40:43,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zkpd.c.gcp-upstream.internal';"
2026-08-11 07:40:44,152 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 07:40:44,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 07:40:44,412 [I] DB_PATCHES have already been applied
I20260811 07:40:44.418203 18723 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-admin:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskIvL2Zc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 07:40:44.418263 18723 mini_ranger.cc:193] Using host: 127.18.72.212
I20260811 07:40:44.422657 18723 mini_ranger.cc:227] Ranger admin URL: http://127.18.72.212:50947
Aug 11, 2026 7:40:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:40:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:40:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 7:40:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 7:40:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 7:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 7:40:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 07:40:45.591820 18723 mini_ranger.cc:160] Time spent starting Ranger: real 8.434s user 0.001s sys 0.010s
Aug 11, 2026 7:41:05 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:41:05 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 7:41:05 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:41:05 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:41:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:41:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:41:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:41:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:41:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:41:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:41:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:41:06 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 07:41:07.497179 18723 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.72.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.72.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/rangerkms_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.72.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/rangerkms_127.18.72.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.72.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.72.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.72.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/HTTP_127.18.72.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.72.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/HTTP_127.18.72.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 07:41:07 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434067, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 07:41:07.605078 18723 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 07:41:07.615976 18723 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 07:41:08.444696 18723 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 07:41:08.444770 18723 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms
I20260811 07:41:08.444782 18723 mini_ranger_kms.cc:227] Using postgres at 127.18.72.212:54171
2026-08-11 07:41:08,618 [I] DB FLAVOR :POSTGRES
2026-08-11 07:41:08,618 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 07:41:08,618 [I] Checking connection
2026-08-11 07:41:08,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 07:41:08,846 [I] connection success
2026-08-11 07:41:08,846 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 07:41:08,846 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 07:41:08,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 07:41:09,089 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 07:41:09,090 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 07:41:09,090 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 07:41:09,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.72.212:54171/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 07:41:09,360 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 07:41:09.825899 18723 mini_ranger_kms.cc:310] Created kms service
I20260811 07:41:10.006934 18723 mini_ranger_kms.cc:326] Created kudu user
I20260811 07:41:10.050163 18723 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 07:41:10 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434067, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:10.440850 18723 mini_ranger_kms.cc:384] Added ranger policy
I20260811 07:41:10.441052 18723 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms:/tmp/dist-test-taskIvL2Zc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskIvL2Zc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 07:41:10.441074 18723 mini_ranger_kms.cc:240] Using host: 127.18.72.212
I20260811 07:41:10.445883 18723 mini_ranger_kms.cc:276] Ranger KMS PID: 32058
I20260811 07:41:10.445978 18723 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.72.212:55831
Aug 11, 2026 7:41:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:41:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:41:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:41:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:41:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:41:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 07:41:12 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786434072, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.72.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:12.866020 18723 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.261s user 0.015s sys 0.143s
I20260811 07:41:12.866156 18723 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 07:41:12.866236 18723 mini_ranger_kms.cc:401] 127.18.72.212:55831/kms/v1/keys
Aug 11 07:41:13 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434072, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.72.212@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
Aug 11 07:41:13 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434072, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.72.212@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
Aug 11 07:41:15 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:41:15.859302 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45109
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:45109
--ranger_config_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:55831/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:15.972078 32110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:15.972368 32110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:15.972440 32110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:15.976395 32110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:15.976475 32110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:15.976512 32110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:15.976564 32110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:15.976605 32110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:15.981783 32110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.72.212:55831/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:45109
--ranger_config_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:15.982774 32110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:15.983757 32110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:15.991870 32115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:15.991871 32116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:15.992002 32110 server_base.cc:1034] running on GCE node
W20260811 07:41:15.991847 32119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:15.992727 32110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:15.993803 32110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:15.994987 32110 hybrid_clock.cc:648] HybridClock initialized: now 1786434075994963 us; error 39 us; skew 500 ppm
Aug 11 07:41:15 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434075, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:16.001529 32110 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:41:16.002384 32110 webserver.cc:460] Webserver started at http://127.18.72.254:43291/ using document root <none> and password file <none>
I20260811 07:41:16.002907 32110 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:16.002979 32110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:16.003214 32110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:41:16 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434075, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:16.064850 32110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "1bcbfcc5f52b40e28f43cc2f8169d560"
format_stamp: "Formatted at 2026-08-11 07:41:16 on dist-test-slave-zkpd"
server_key: "0d6443e06133d39b6771ed51b41f1705"
server_key_iv: "46cffe83a648c3a9457395859575f2e7"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:16.065501 32110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "1bcbfcc5f52b40e28f43cc2f8169d560"
format_stamp: "Formatted at 2026-08-11 07:41:16 on dist-test-slave-zkpd"
server_key: "0d6443e06133d39b6771ed51b41f1705"
server_key_iv: "46cffe83a648c3a9457395859575f2e7"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:16.069530 32110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 07:41:16.086836 32128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:16.088654 32110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:41:16.088827 32110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "1bcbfcc5f52b40e28f43cc2f8169d560"
format_stamp: "Formatted at 2026-08-11 07:41:16 on dist-test-slave-zkpd"
server_key: "0d6443e06133d39b6771ed51b41f1705"
server_key_iv: "46cffe83a648c3a9457395859575f2e7"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:16.088986 32110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:16.101907 32110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:16.104982 32110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:16.105181 32110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:16.113492 32110 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:45109
I20260811 07:41:16.113566 32185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:45109 every 8 connection(s)
I20260811 07:41:16.114487 32110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:41:16.116763 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 32110
I20260811 07:41:16.116905 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:41:16.117193 18723 external_mini_cluster.cc:1428] Setting key 274e69ca4b19f9b14d5bc77b9e353d2f
I20260811 07:41:16.118485 32186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 07:41:16 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:16.126585 32186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Bootstrap starting.
I20260811 07:41:16.129667 32186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:16.130662 32189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:16.118644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.18.72.254:45109)
0811 07:41:16.119051 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:16.119075 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:16.119111 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:41:16.119744 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:16.119759 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:16.120143 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:16.120488 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:16.121281 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:16.122254 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:16.123070 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:16.123521 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:16.123683 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:16.126134 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:16.126139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:16.126152 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:16.126178 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:16.127826 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:16.128328 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:16.128334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:16.128338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:16.128441 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:16.128720 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:16.128723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:16.128725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:16.129119 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:16.129323 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:16.129618 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 07:41:16.129940 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":137,"threads_started":1}
I20260811 07:41:16.131253 32186 log.cc:824] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:16.133327 32186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: No bootstrap required, opened a new log
I20260811 07:41:16.136401 32186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } }
I20260811 07:41:16.136636 32186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:16.136730 32186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcbfcc5f52b40e28f43cc2f8169d560, State: Initialized, Role: FOLLOWER
I20260811 07:41:16.137207 32186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } }
I20260811 07:41:16.137387 32186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:41:16.137458 32186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:41:16.137557 32186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:16.138440 32186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } }
I20260811 07:41:16.138784 32186 leader_election.cc:302] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bcbfcc5f52b40e28f43cc2f8169d560; no voters:
I20260811 07:41:16.139098 32186 leader_election.cc:288] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:41:16.139295 32191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:16.139582 32191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcbfcc5f52b40e28f43cc2f8169d560, State: Running, Role: LEADER
I20260811 07:41:16.139936 32191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } }
I20260811 07:41:16.140422 32186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:16.141201 32191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bcbfcc5f52b40e28f43cc2f8169d560. Latest consensus state: current_term: 1 leader_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } } }
I20260811 07:41:16.141331 32191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:16.142028 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } } }
I20260811 07:41:16.142205 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:16.142733 32186 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 07:41:16.144392 32201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:41:16.150851 32201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:41:16.158398 32201 catalog_manager.cc:1347] Generated new cluster ID: 325ab2b34c2f453d8625161afaf99f2a
I20260811 07:41:16.158552 32201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:41:16.188704 32201 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:41:16.189656 32201 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:41:16.197156 32201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Generated new TSK 0
I20260811 07:41:16.198094 32201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:41:16.900799 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:55831/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45109
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:17.028529 32236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:17.028795 32236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:17.028910 32236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:17.033011 32236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:17.033078 32236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:17.033156 32236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:41:17.038275 32236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.72.212:55831/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:17.039270 32236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:17.040419 32236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:17.051198 32242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:17.051697 32243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:17.051198 32245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:17.052759 32236 server_base.cc:1034] running on GCE node
I20260811 07:41:17.053149 32236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:17.053980 32236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:17.057261 32236 hybrid_clock.cc:648] HybridClock initialized: now 1786434077057226 us; error 41 us; skew 500 ppm
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:17.060233 32236 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:41:17.061084 32236 webserver.cc:460] Webserver started at http://127.18.72.193:44923/ using document root <none> and password file <none>
I20260811 07:41:17.061645 32236 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:17.061731 32236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:17.061949 32236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:17.080862 32236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "fe0af2c0de58472eb25a601e29afc340"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "e839fcd8cebc5ee93fd7a47dea1f756e"
server_key_iv: "4f863d95cf6cb2201f3a119fa79b9cb7"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.081667 32236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "fe0af2c0de58472eb25a601e29afc340"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "e839fcd8cebc5ee93fd7a47dea1f756e"
server_key_iv: "4f863d95cf6cb2201f3a119fa79b9cb7"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.086731 32236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 07:41:17.104664 32255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.106535 32236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:41:17.106745 32236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "fe0af2c0de58472eb25a601e29afc340"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "e839fcd8cebc5ee93fd7a47dea1f756e"
server_key_iv: "4f863d95cf6cb2201f3a119fa79b9cb7"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.106990 32236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:17.125231 32236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:17.128873 32236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:17.129110 32236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:17.129928 32236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:17.131045 32236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:17.131151 32236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.131290 32236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:17.131379 32236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.144299 32236 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:45577
I20260811 07:41:17.145407 32236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:41:17.150591 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 32236
I20260811 07:41:17.150717 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:41:17.151059 18723 external_mini_cluster.cc:1428] Setting key c213d6f2e49674c315fd8e57c0355f44
I20260811 07:41:17.146961 32374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:45577 every 8 connection(s)
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:17.164451 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:17.149484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45109 (local address 127.18.72.193:33837)
0811 07:41:17.150044 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:17.150113 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:17.151224 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:17.152541 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:17.152568 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:17.152932 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:17.153644 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:17.153669 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.154890 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.154896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:17.155819 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:17.155829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.156583 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.156589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:17.156818 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:17.157508 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:17.157533 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:17.159040 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:17.161262 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:17.161269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:17.161284 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:17.161626 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:17.161941 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:17.161944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:17.161946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:17.162118 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:17.162594 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:17.162602 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:17.162920 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 07:41:17.163097 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 07:41:17.163437 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":170,"threads_started":1}
I20260811 07:41:17.165987 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:17.150624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:33837 (local address 127.18.72.254:45109)
0811 07:41:17.151073 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:17.151083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:17.151224 (+ 141us) server_negotiation.cc:407] Connection header received
0811 07:41:17.151479 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:17.151483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:17.151543 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:17.151661 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:17.153853 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.154718 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.155990 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.156399 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.157168 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:17.159208 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:17.159212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:17.159219 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:17.159242 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:17.161112 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:17.161757 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:17.161760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:17.161762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:17.161846 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:17.162259 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:17.162263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:17.162265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:17.162476 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:17.165598 (+ 3122us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:17.165679 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 07:41:17.165808 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":112,"threads_started":1}
I20260811 07:41:17.167219 32375 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45109
I20260811 07:41:17.167464 32375 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:17.168139 32375 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
I20260811 07:41:17.169950 32147 ts_manager.cc:194] Registered new tserver with Master: fe0af2c0de58472eb25a601e29afc340 (127.18.72.193:45577)
I20260811 07:41:17.171489 32147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:33837
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:41:17.192915 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:55831/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45109
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 07:41:17.354189 32385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:17.354589 32385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:17.354667 32385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:17.360553 32385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:17.360625 32385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:17.360709 32385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:41:17.368458 32385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.72.212:55831/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:17.369745 32385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:17.371088 32385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:17.384027 32396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:17.384035 32393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:17.384521 32394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:17.385345 32385 server_base.cc:1034] running on GCE node
I20260811 07:41:17.385807 32385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:17.386590 32385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:17.387787 32385 hybrid_clock.cc:648] HybridClock initialized: now 1786434077387757 us; error 45 us; skew 500 ppm
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:17.391696 32385 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:41:17.392802 32385 webserver.cc:460] Webserver started at http://127.18.72.194:42075/ using document root <none> and password file <none>
I20260811 07:41:17.393544 32385 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:17.393646 32385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:17.393914 32385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:17.412272 32385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "95fcf803849144e49a662597c54226db"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "8f861f8e79570d4b84a9aeb6d917c2d8"
server_key_iv: "2f35fe30228550ccef529751715adadb"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.413030 32385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "95fcf803849144e49a662597c54226db"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "8f861f8e79570d4b84a9aeb6d917c2d8"
server_key_iv: "2f35fe30228550ccef529751715adadb"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.418128 32385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 07:41:17.436837 32403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.438295 32385 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260811 07:41:17.438444 32385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "95fcf803849144e49a662597c54226db"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "8f861f8e79570d4b84a9aeb6d917c2d8"
server_key_iv: "2f35fe30228550ccef529751715adadb"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.438575 32385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:17.457974 32385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:17.466809 32385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:17.467011 32385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:17.467693 32385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:17.468673 32385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:17.468748 32385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.468827 32385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:17.468849 32385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.480234 32385 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:44513
I20260811 07:41:17.480345 32522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:44513 every 8 connection(s)
I20260811 07:41:17.481386 32385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:41:17.486888 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 32385
I20260811 07:41:17.487015 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:41:17.487317 18723 external_mini_cluster.cc:1428] Setting key a5ac35a4537d2761ae83849cf33de8f2
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:17.500080 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:17.484215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45109 (local address 127.18.72.194:49167)
0811 07:41:17.484818 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:17.484880 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:17.486089 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:17.486746 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:17.486756 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:17.487120 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:17.487923 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:17.487948 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.489106 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.489109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:17.490454 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:17.490469 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.491310 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.491316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:17.491530 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:17.492232 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:17.492267 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:17.494069 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:17.496410 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:17.496417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:17.496435 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:17.496862 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:17.497208 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:17.497214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:17.497219 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:17.497415 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:17.498009 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:17.498018 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:17.498368 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 07:41:17.498590 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 07:41:17.498948 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":161,"threads_started":1}
I20260811 07:41:17.500721 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:17.483840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:49167 (local address 127.18.72.254:45109)
0811 07:41:17.484043 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:17.484049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:17.484993 (+ 944us) server_negotiation.cc:407] Connection header received
0811 07:41:17.486347 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:17.486351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:17.486407 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:17.486512 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:17.488126 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.488948 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.490603 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.491004 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.491094 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:17.494233 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:17.494237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:17.494242 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:17.494277 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:17.496268 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:17.496994 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:17.497000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:17.497003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:17.497099 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:17.497544 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:17.497549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:17.497553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:17.497834 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:17.497945 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:17.500435 (+ 2490us) server_negotiation.cc:299] Negotiation successful
0811 07:41:17.500577 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 07:41:17.501983 32523 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45109
I20260811 07:41:17.502242 32523 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:17.502828 32523 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:41:17.504108 32147 ts_manager.cc:194] Registered new tserver with Master: 95fcf803849144e49a662597c54226db (127.18.72.194:44513)
I20260811 07:41:17.504681 32147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:49167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:41:17.526451 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:55831/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45109
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:17.668893 32529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:17.669260 32529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:17.669344 32529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:17.673607 32529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:17.673683 32529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:17.673795 32529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:41:17.680970 32529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.72.212:55831/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:17.682032 32529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:17.683054 32529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:17.696364 32535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:17.696825 32538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:17.696338 32536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:17.697134 32529 server_base.cc:1034] running on GCE node
I20260811 07:41:17.697525 32529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:17.698163 32529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:17.699347 32529 hybrid_clock.cc:648] HybridClock initialized: now 1786434077699288 us; error 73 us; skew 500 ppm
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:17.702337 32529 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:41:17.703256 32529 webserver.cc:460] Webserver started at http://127.18.72.195:42607/ using document root <none> and password file <none>
I20260811 07:41:17.703801 32529 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:17.703886 32529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:17.704118 32529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:17.725800 32529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "69819e4ca77347c7afce90c7e3205323"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "d6614e0cf7ca52e25f121c33a579c8aa"
server_key_iv: "c2beb95288652ed9a57943f3d56ff1b9"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.726477 32529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "69819e4ca77347c7afce90c7e3205323"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "d6614e0cf7ca52e25f121c33a579c8aa"
server_key_iv: "c2beb95288652ed9a57943f3d56ff1b9"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.731779 32529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 07:41:17.758139 32546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.766350 32529 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.001s
I20260811 07:41:17.766500 32529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "69819e4ca77347c7afce90c7e3205323"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "d6614e0cf7ca52e25f121c33a579c8aa"
server_key_iv: "c2beb95288652ed9a57943f3d56ff1b9"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:17.766644 32529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:17.807943 32529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:17.811451 32529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:17.811630 32529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:17.812281 32529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:17.813256 32529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:17.813320 32529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.813385 32529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:17.813414 32529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:17.822890 32529 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:39101
I20260811 07:41:17.823994 32529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:41:17.826280 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 32529
I20260811 07:41:17.826411 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:41:17.826719 18723 external_mini_cluster.cc:1428] Setting key fc4b6426dde078c8753836198f53e280
I20260811 07:41:17.832297 32665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:39101 every 8 connection(s)
Aug 11 07:41:17 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434077, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:17.858848 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:17.834062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:33109 (local address 127.18.72.254:45109)
0811 07:41:17.834191 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:17.834195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:17.835109 (+ 914us) server_negotiation.cc:407] Connection header received
0811 07:41:17.836195 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:17.836199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:17.836256 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:17.836367 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:17.837443 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.838056 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.841958 (+ 3902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.842441 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.842596 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:17.848205 (+ 5609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:17.848209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:17.848214 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:17.848246 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:17.851315 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:17.856980 (+ 5665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:17.856984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:17.856986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:17.857088 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:17.857513 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:17.857516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:17.857519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:17.857752 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:17.857846 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:17.858556 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 07:41:17.858682 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:41:17.860574 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:17.834453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45109 (local address 127.18.72.195:33109)
0811 07:41:17.834962 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:17.835016 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:17.835998 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:17.836541 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:17.836549 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:17.836824 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:17.837306 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:17.837320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.838216 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.838220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:17.841783 (+ 3563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:17.841807 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:17.842851 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:17.842856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:17.843120 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:17.844590 (+ 1470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:17.844613 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:17.848012 (+ 3399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:17.856401 (+ 8389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:17.856408 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:17.856428 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:17.856846 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:17.857202 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:17.857205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:17.857208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:17.857403 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:17.857916 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:17.857924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:17.858266 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 07:41:17.859103 (+ 837us) client_negotiation.cc:240] Negotiation successful
0811 07:41:17.859465 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":149,"threads_started":1}
I20260811 07:41:17.861786 32666 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45109
I20260811 07:41:17.862052 32666 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:17.862625 32666 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
I20260811 07:41:17.864470 32145 ts_manager.cc:194] Registered new tserver with Master: 69819e4ca77347c7afce90c7e3205323 (127.18.72.195:39101)
I20260811 07:41:17.865221 32145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:33109
I20260811 07:41:17.865962 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:41:17.866665 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 32236
I20260811 07:41:17.882593 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 32385
I20260811 07:41:17.893285 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 32529
I20260811 07:41:17.904536 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 32110
I20260811 07:41:17.915263 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45109
--webserver_interface=127.18.72.254
--webserver_port=43291
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:45109
--ranger_config_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:55831/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.72.212:55831/kms with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:18.028020 32669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:18.028313 32669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:18.028379 32669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:18.032467 32669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:18.032539 32669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:18.032577 32669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:18.032626 32669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:18.032665 32669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:18.038184 32669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.72.212:55831/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:45109
--ranger_config_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=43291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:18.039229 32669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:18.040302 32669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:18.048655 32675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:18.048763 32676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:18.048655 32678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:18.049569 32669 server_base.cc:1034] running on GCE node
I20260811 07:41:18.050244 32669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:18.051695 32669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:18.052910 32669 hybrid_clock.cc:648] HybridClock initialized: now 1786434078052875 us; error 41 us; skew 500 ppm
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:18.056277 32669 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:41:18.057212 32669 webserver.cc:460] Webserver started at http://127.18.72.254:43291/ using document root <none> and password file <none>
I20260811 07:41:18.057742 32669 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:18.057806 32669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:18.141300 32669 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 07:41:18.144938 32686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.146485 32669 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 07:41:18.146773 32669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "1bcbfcc5f52b40e28f43cc2f8169d560"
format_stamp: "Formatted at 2026-08-11 07:41:16 on dist-test-slave-zkpd"
server_key: "0d6443e06133d39b6771ed51b41f1705"
server_key_iv: "46cffe83a648c3a9457395859575f2e7"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:18.147367 32669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:18.161538 32669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:18.164531 32669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:18.164968 32669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:18.172977 32669 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:45109
I20260811 07:41:18.172983 32743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:45109 every 8 connection(s)
I20260811 07:41:18.174129 32669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:41:18.178508 32744 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:41:18.180207 32744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Bootstrap starting.
I20260811 07:41:18.183626 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 32669
I20260811 07:41:18.184163 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:55831/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:45577
--local_ip_for_outbound_sockets=127.18.72.193
--tserver_master_addrs=127.18.72.254:45109
--webserver_port=44923
--webserver_interface=127.18.72.193
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.72.212:55831/kms with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:41:18.201964 32744 log.cc:824] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:18.210789 32744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:41:18.211239 32744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Bootstrap complete.
I20260811 07:41:18.217067 32744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } }
I20260811 07:41:18.217566 32744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcbfcc5f52b40e28f43cc2f8169d560, State: Initialized, Role: FOLLOWER
I20260811 07:41:18.218128 32744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } }
I20260811 07:41:18.218274 32744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:41:18.218341 32744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:41:18.218439 32744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:41:18.220912 32744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } }
I20260811 07:41:18.221278 32744 leader_election.cc:302] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bcbfcc5f52b40e28f43cc2f8169d560; no voters:
I20260811 07:41:18.221622 32744 leader_election.cc:288] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:41:18.221807 32748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:41:18.222088 32748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [term 2 LEADER]: Becoming Leader. State: Replica: 1bcbfcc5f52b40e28f43cc2f8169d560, State: Running, Role: LEADER
I20260811 07:41:18.222442 32748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } }
I20260811 07:41:18.222901 32744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:18.223531 32748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } } }
I20260811 07:41:18.223709 32748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:18.223996 32748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bcbfcc5f52b40e28f43cc2f8169d560. Latest consensus state: current_term: 2 leader_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bcbfcc5f52b40e28f43cc2f8169d560" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45109 } } }
I20260811 07:41:18.224125 32748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:18.224411 32756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:41:18.227566 32756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:41:18.228117 32744 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 07:41:18.228338 32756 catalog_manager.cc:1259] Loaded cluster ID: 325ab2b34c2f453d8625161afaf99f2a
I20260811 07:41:18.228427 32756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:41:18.230937 32756 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:41:18.231606 32756 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Loaded TSK: 0
I20260811 07:41:18.232877 32756 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:41:18.340021 32746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:18.340466 32746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:18.340610 32746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:18.346247 32746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:18.346328 32746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:18.346424 32746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:41:18.352178 32746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.72.212:55831/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:45577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=44923
--tserver_master_addrs=127.18.72.254:45109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:18.353256 32746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:18.354291 32746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:18.365155 316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:18.365370 314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:18.365679 313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:18.365942 32746 server_base.cc:1034] running on GCE node
I20260811 07:41:18.366389 32746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:18.367203 32746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:18.371186 32746 hybrid_clock.cc:648] HybridClock initialized: now 1786434078371153 us; error 40 us; skew 500 ppm
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:18.375278 32746 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:41:18.376477 32746 webserver.cc:460] Webserver started at http://127.18.72.193:44923/ using document root <none> and password file <none>
I20260811 07:41:18.377286 32746 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:18.377442 32746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:18.397761 32746 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 07:41:18.402194 325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.403707 32746 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 07:41:18.403820 32746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "fe0af2c0de58472eb25a601e29afc340"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "e839fcd8cebc5ee93fd7a47dea1f756e"
server_key_iv: "4f863d95cf6cb2201f3a119fa79b9cb7"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:18.404567 32746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:18.449191 32746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:18.453195 32746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:18.453419 32746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:18.454142 32746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:18.455241 32746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:18.455291 32746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.455343 32746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:18.455364 32746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.466904 32746 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:45577
I20260811 07:41:18.468266 32746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:41:18.474608 448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:45577 every 8 connection(s)
I20260811 07:41:18.476799 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 32746
I20260811 07:41:18.477231 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:55831/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:44513
--local_ip_for_outbound_sockets=127.18.72.194
--tserver_master_addrs=127.18.72.254:45109
--webserver_port=42075
--webserver_interface=127.18.72.194
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.72.212:55831/kms with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:18.496491 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:18.476955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45109 (local address 127.18.72.193:60767)
0811 07:41:18.477467 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:18.477520 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:18.478515 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:18.481083 (+ 2568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:18.481092 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:18.481485 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:18.482153 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:18.482178 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.484336 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.484343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:18.485874 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:18.485889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.487112 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.487119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:18.487377 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:18.488135 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:18.488163 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:18.490040 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:18.492879 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:18.492888 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:18.492905 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:18.493285 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:18.493703 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:18.493706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:18.493708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:18.493898 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:18.494625 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:18.494632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:18.494917 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 07:41:18.495099 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 07:41:18.495437 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":157,"threads_started":1}
I20260811 07:41:18.498785 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:18.479325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:60767 (local address 127.18.72.254:45109)
0811 07:41:18.479860 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:18.479877 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:18.479964 (+ 87us) server_negotiation.cc:407] Connection header received
0811 07:41:18.480274 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:18.480291 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:18.480675 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:18.481008 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:18.482375 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.484144 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.486069 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.486911 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.487820 (+ 909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:18.490307 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:18.490316 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:18.490334 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:18.490369 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:18.492728 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:18.493434 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:18.493441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:18.493446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:18.493577 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:18.494017 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:18.494020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:18.494022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:18.494490 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:18.497489 (+ 2999us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:18.497645 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 07:41:18.497930 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":128,"threads_started":1}
I20260811 07:41:18.501768 449 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45109
I20260811 07:41:18.502007 449 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:18.502578 449 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
W20260811 07:41:18.504876 449 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:45109 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 07:41:18.505007 449 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:18.505163 449 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
I20260811 07:41:18.506219 449 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:18.506392 449 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
W20260811 07:41:18.507160 449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:41:18.636318 453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:18.636619 453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:18.636687 453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:18.641010 453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:18.641081 453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:18.641167 453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:41:18.647524 453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.72.212:55831/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:44513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=42075
--tserver_master_addrs=127.18.72.254:45109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:18.648765 453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:18.650038 453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:18.659967 459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:18.659965 460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:18.660043 462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:18.660461 453 server_base.cc:1034] running on GCE node
I20260811 07:41:18.660866 453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:18.661486 453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:18.662672 453 hybrid_clock.cc:648] HybridClock initialized: now 1786434078662647 us; error 42 us; skew 500 ppm
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:18.665680 453 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:41:18.666642 453 webserver.cc:460] Webserver started at http://127.18.72.194:42075/ using document root <none> and password file <none>
I20260811 07:41:18.667219 453 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:18.667313 453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:18.689919 453 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260811 07:41:18.696763 470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.698478 453 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260811 07:41:18.698663 453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "95fcf803849144e49a662597c54226db"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "8f861f8e79570d4b84a9aeb6d917c2d8"
server_key_iv: "2f35fe30228550ccef529751715adadb"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:18.699483 453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:18.721740 453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:18.725844 453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:18.726054 453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:18.726748 453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:18.727816 453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:18.727878 453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.727974 453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:18.728006 453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.739003 453 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:44513
I20260811 07:41:18.740593 591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:44513 every 8 connection(s)
I20260811 07:41:18.741035 453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:41:18.747402 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 453
I20260811 07:41:18.747920 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.72.212:55831/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:39101
--local_ip_for_outbound_sockets=127.18.72.195
--tserver_master_addrs=127.18.72.254:45109
--webserver_port=42607
--webserver_interface=127.18.72.195
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.72.212:55831/kms with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:18.758448 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:18.743432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45109 (local address 127.18.72.194:55497)
0811 07:41:18.743924 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:18.743976 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:18.744848 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:18.745392 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:18.745399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:18.745653 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:18.746119 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:18.746132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.747301 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.747307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:18.748736 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:18.748752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.749444 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.749451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:18.749683 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:18.750388 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:18.750404 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:18.752068 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:18.754352 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:18.754358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:18.754375 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:18.754735 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:18.755867 (+ 1132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:18.755870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:18.755872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:18.756066 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:18.756566 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:18.756572 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:18.756917 (+ 345us) client_negotiation.cc:769] Sending connection context
0811 07:41:18.757114 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 07:41:18.757430 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":133,"threads_started":1}
I20260811 07:41:18.759467 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:18.742992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:55497 (local address 127.18.72.254:45109)
0811 07:41:18.743118 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:18.743123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:18.744072 (+ 949us) server_negotiation.cc:407] Connection header received
0811 07:41:18.745066 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:18.745070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:18.745133 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:18.745240 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:18.746275 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.747144 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.748880 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.749260 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.749389 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:18.752234 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:18.752238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:18.752240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:18.752270 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:18.754208 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:18.755639 (+ 1431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:18.755642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:18.755644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:18.755738 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:18.756186 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:18.756192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:18.756194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:18.756413 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:18.756509 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:18.759176 (+ 2667us) server_negotiation.cc:299] Negotiation successful
0811 07:41:18.759317 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:41:18.760480 592 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45109
I20260811 07:41:18.760711 592 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:18.761255 592 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
W20260811 07:41:18.762857 592 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:45109 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 07:41:18.762990 592 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:18.763183 592 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
I20260811 07:41:18.764005 592 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:18.764153 592 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
W20260811 07:41:18.764884 592 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:41:18.894974 595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:18.895303 595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:18.895380 595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:18.901221 595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:18.901288 595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:18.901381 595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:41:18.906849 595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.72.212:55831/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:39101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=42607
--tserver_master_addrs=127.18.72.254:45109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:18.907950 595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:18.909053 595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:18.919914 604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:18.920082 595 server_base.cc:1034] running on GCE node
W20260811 07:41:18.920140 607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:18.919942 603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:18.920616 595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:18.921240 595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:18.922410 595 hybrid_clock.cc:648] HybridClock initialized: now 1786434078922390 us; error 35 us; skew 500 ppm
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:18.925414 595 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:41:18.926221 595 webserver.cc:460] Webserver started at http://127.18.72.195:42607/ using document root <none> and password file <none>
I20260811 07:41:18.926887 595 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:18.927006 595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 07:41:18 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:18.952265 595 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 07:41:18.955392 615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.956988 595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:41:18.957129 595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "69819e4ca77347c7afce90c7e3205323"
format_stamp: "Formatted at 2026-08-11 07:41:17 on dist-test-slave-zkpd"
server_key: "d6614e0cf7ca52e25f121c33a579c8aa"
server_key_iv: "c2beb95288652ed9a57943f3d56ff1b9"
server_key_version: "kuduclusterkey@0"
I20260811 07:41:18.957824 595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:18.971628 595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:18.974741 595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:18.974941 595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:18.975561 595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:18.976608 595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:18.976670 595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.976730 595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:18.976763 595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:18.987286 595 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:39101
I20260811 07:41:18.988554 595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:41:18.989527 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 595
I20260811 07:41:19.000293 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:18.990695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48514 (local address 127.18.72.254:45109)
0811 07:41:18.990852 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:18.990857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:18.990877 (+ 20us) server_negotiation.cc:407] Connection header received
0811 07:41:18.991087 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:18.991091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:18.991151 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:18.991252 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:18.992172 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.993000 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.994253 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:18.994624 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:18.994734 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:18.996540 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:18.996544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:18.996546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:18.996574 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:18.998393 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:18.998887 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:18.998891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:18.998893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:18.998982 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:18.999268 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:18.999271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:18.999273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:18.999517 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:18.999738 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:18.999979 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 07:41:19.000111 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:41:19.001979 736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:39101 every 8 connection(s)
Aug 11 07:41:19 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:19.030675 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.004030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45109 (local address 127.18.72.195:41921)
0811 07:41:19.004593 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:19.004647 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:19.005546 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:19.006111 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:19.006118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:19.006430 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:19.006921 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:19.006934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.008009 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.008012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:19.009003 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:19.009013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.009658 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.009665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:19.009848 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.010437 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:19.010474 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:19.021669 (+ 11195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:19.025191 (+ 3522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:19.025200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:19.025220 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:19.025531 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:19.025895 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:19.025898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:19.025900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:19.028134 (+ 2234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:19.028666 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:19.028673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:19.029041 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 07:41:19.029253 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 07:41:19.029611 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":167,"threads_started":1}
I20260811 07:41:19.031430 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.004088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:41921 (local address 127.18.72.254:45109)
0811 07:41:19.004220 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:19.004225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:19.004761 (+ 536us) server_negotiation.cc:407] Connection header received
0811 07:41:19.005740 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:19.005743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:19.005796 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:19.005913 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:19.007067 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.007842 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.009127 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.009502 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.009622 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.021845 (+ 12223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:19.021851 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:19.021854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:19.021881 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:19.025042 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:19.025652 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:19.025656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:19.025658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:19.025749 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:19.028280 (+ 2531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:19.028284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:19.028285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:19.028538 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:19.028644 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:19.031147 (+ 2503us) server_negotiation.cc:299] Negotiation successful
0811 07:41:19.031280 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:41:19.032457 737 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45109
I20260811 07:41:19.032706 737 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:19.033316 737 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
I20260811 07:41:19.034638 32703 ts_manager.cc:194] Registered new tserver with Master: 69819e4ca77347c7afce90c7e3205323 (127.18.72.195:39101)
I20260811 07:41:19.035771 32703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:41921
Aug 11 07:41:19 dist-test-slave-zkpd krb5kdc[31645](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786434079, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:19.507387 449 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:19.507690 449 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
I20260811 07:41:19.508404 32703 ts_manager.cc:194] Registered new tserver with Master: fe0af2c0de58472eb25a601e29afc340 (127.18.72.193:45577)
I20260811 07:41:19.509075 32703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:60767
Aug 11 07:41:19 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434079, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:19.765120 592 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:19.765426 592 heartbeater.cc:511] Master 127.18.72.254:45109 requested a full tablet report, sending...
I20260811 07:41:19.766119 32703 ts_manager.cc:194] Registered new tserver with Master: 95fcf803849144e49a662597c54226db (127.18.72.194:44513)
I20260811 07:41:19.766700 32703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:55497
I20260811 07:41:19.777225 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:41:19.791738 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.778802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48520 (local address 127.18.72.254:45109)
0811 07:41:19.780362 (+ 1560us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:19.780367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:19.780614 (+ 247us) server_negotiation.cc:407] Connection header received
0811 07:41:19.780867 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:19.780870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:19.780927 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:19.780992 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:19.782029 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.782827 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.784013 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.784359 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.784439 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.787482 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:19.787486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:19.787489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:19.787516 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:19.789760 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:19.790282 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:19.790286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:19.790289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:19.790388 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:19.790719 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:19.790723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:19.790725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:19.790945 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:19.791051 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:19.791426 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 07:41:19.791555 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1474,"thread_start_us":84,"threads_started":1}
I20260811 07:41:19.796366 32703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:41:19.801141 32703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:41:19.819295 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.813410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.18.72.195:39101)
0811 07:41:19.813952 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:19.813960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:19.814937 (+ 977us) server_negotiation.cc:407] Connection header received
0811 07:41:19.815150 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:19.815156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:19.815372 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:19.815821 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:19.815982 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.817013 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.818105 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.818673 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.818803 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.818943 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:19.819072 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 07:41:19.819154 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":63,"threads_started":1}
I20260811 07:41:19.819301 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.813142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:39101 (local address 127.0.0.1:57626)
0811 07:41:19.814777 (+ 1635us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:19.814821 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:19.814986 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:19.815531 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:19.815538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:19.815564 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:19.815687 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:19.815692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.817130 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.817133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:19.817972 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:19.817981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.818789 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.818793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:19.818895 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.818930 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 07:41:19.819043 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 07:41:19.819155 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1467,"thread_start_us":106,"threads_started":1}
I20260811 07:41:19.822628 665 tablet_service.cc:1467] Processing CreateTablet for tablet b68d5e052eea479ca6095198a9a73fc2 (DEFAULT_TABLE table=test-table [id=cd98583b701f42e99dcca51de73941ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:19.823832 665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68d5e052eea479ca6095198a9a73fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:19.827397 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.813142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:45577 (local address 127.0.0.1:55588)
0811 07:41:19.815222 (+ 2080us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:19.815234 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:19.815317 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:19.820391 (+ 5074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:19.820394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:19.820412 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:19.820512 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:19.820518 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.822250 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.822253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:19.823444 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:19.823458 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.826990 (+ 3532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.826993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:19.827093 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.827119 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:41:19.827171 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 07:41:19.827232 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2023,"mutex_wait_us":865,"thread_start_us":91,"threads_started":1}
I20260811 07:41:19.828882 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.813730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55588 (local address 127.18.72.193:45577)
0811 07:41:19.819788 (+ 6058us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:19.819795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:19.819821 (+ 26us) server_negotiation.cc:407] Connection header received
0811 07:41:19.819916 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:19.819921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:19.820106 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:19.820251 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:19.820647 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.821694 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.824420 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.826848 (+ 2428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.827676 (+ 828us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.827829 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:19.827884 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 07:41:19.827997 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5936,"thread_start_us":125,"threads_started":1}
I20260811 07:41:19.831842 357 tablet_service.cc:1467] Processing CreateTablet for tablet b68d5e052eea479ca6095198a9a73fc2 (DEFAULT_TABLE table=test-table [id=cd98583b701f42e99dcca51de73941ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:19.832370 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.814382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:44513 (local address 127.0.0.1:38106)
0811 07:41:19.819470 (+ 5088us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:19.819482 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:19.819559 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:19.824742 (+ 5183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:19.824746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:19.824764 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:19.824842 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:19.824848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.829733 (+ 4885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.829736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:19.830799 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:19.830811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.831993 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.831997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:19.832081 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.832107 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:41:19.832156 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 07:41:19.832225 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5038,"thread_start_us":86,"threads_started":1}
I20260811 07:41:19.833006 357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68d5e052eea479ca6095198a9a73fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:19.833081 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:19.814534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.18.72.194:44513)
0811 07:41:19.815543 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:19.815550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:19.821817 (+ 6267us) server_negotiation.cc:407] Connection header received
0811 07:41:19.821898 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:19.821903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:19.822095 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:19.824985 (+ 2890us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:19.825123 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.826160 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.830965 (+ 4805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:19.831751 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:19.832634 (+ 883us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:19.832793 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:19.832845 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 07:41:19.832939 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":91,"threads_started":1}
I20260811 07:41:19.835503 501 tablet_service.cc:1467] Processing CreateTablet for tablet b68d5e052eea479ca6095198a9a73fc2 (DEFAULT_TABLE table=test-table [id=cd98583b701f42e99dcca51de73941ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:19.836496 501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68d5e052eea479ca6095198a9a73fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:19.846386 772 tablet_bootstrap.cc:492] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Bootstrap starting.
I20260811 07:41:19.849726 772 tablet_bootstrap.cc:654] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:19.850718 772 log.cc:824] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:19.851982 771 tablet_bootstrap.cc:492] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Bootstrap starting.
I20260811 07:41:19.852897 774 tablet_bootstrap.cc:492] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340: Bootstrap starting.
I20260811 07:41:19.853631 772 tablet_bootstrap.cc:492] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: No bootstrap required, opened a new log
I20260811 07:41:19.853863 772 ts_tablet_manager.cc:1397] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 07:41:19.854647 771 tablet_bootstrap.cc:654] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:19.854907 774 tablet_bootstrap.cc:654] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:19.855494 771 log.cc:824] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:19.855576 774 log.cc:824] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:19.857558 771 tablet_bootstrap.cc:492] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: No bootstrap required, opened a new log
I20260811 07:41:19.857905 774 tablet_bootstrap.cc:492] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340: No bootstrap required, opened a new log
I20260811 07:41:19.857736 772 raft_consensus.cc:348] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
I20260811 07:41:19.857996 772 raft_consensus.cc:374] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:19.858001 771 ts_tablet_manager.cc:1397] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 07:41:19.858040 772 raft_consensus.cc:729] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fcf803849144e49a662597c54226db, State: Initialized, Role: FOLLOWER
I20260811 07:41:19.858045 774 ts_tablet_manager.cc:1397] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 07:41:19.858465 772 consensus_queue.cc:260] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
I20260811 07:41:19.860489 772 ts_tablet_manager.cc:1428] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 07:41:19.860910 592 heartbeater.cc:503] Master 127.18.72.254:45109 was elected leader, sending a full tablet report...
I20260811 07:41:19.862879 771 raft_consensus.cc:348] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
I20260811 07:41:19.863134 771 raft_consensus.cc:374] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:19.863224 771 raft_consensus.cc:729] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69819e4ca77347c7afce90c7e3205323, State: Initialized, Role: FOLLOWER
I20260811 07:41:19.863685 771 consensus_queue.cc:260] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
I20260811 07:41:19.864477 737 heartbeater.cc:503] Master 127.18.72.254:45109 was elected leader, sending a full tablet report...
I20260811 07:41:19.864748 771 ts_tablet_manager.cc:1428] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260811 07:41:19.865024 774 raft_consensus.cc:348] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
I20260811 07:41:19.865263 774 raft_consensus.cc:374] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:19.865345 774 raft_consensus.cc:729] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0af2c0de58472eb25a601e29afc340, State: Initialized, Role: FOLLOWER
I20260811 07:41:19.865769 774 consensus_queue.cc:260] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
I20260811 07:41:19.866732 774 ts_tablet_manager.cc:1428] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 07:41:19.867079 449 heartbeater.cc:503] Master 127.18.72.254:45109 was elected leader, sending a full tablet report...
W20260811 07:41:19.979058 450 tablet.cc:2369] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:41:19.994196 593 tablet.cc:2369] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:41:20.000964 779 raft_consensus.cc:482] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:20.001180 779 raft_consensus.cc:504] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
Aug 11 07:41:19 dist-test-slave-zkpd krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434079, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for HTTP/127.18.72.212@KRBTEST.COM
I20260811 07:41:20.002660 779 leader_election.cc:288] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69819e4ca77347c7afce90c7e3205323 (127.18.72.195:39101), 95fcf803849144e49a662597c54226db (127.18.72.194:44513)
W20260811 07:41:20.004392 738 tablet.cc:2369] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:41:20.009847 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:20.003264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:51869 (local address 127.18.72.195:39101)
0811 07:41:20.003387 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:20.003393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:20.003799 (+ 406us) server_negotiation.cc:407] Connection header received
0811 07:41:20.003976 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:20.003979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:20.004031 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:20.004106 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:20.005285 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.006020 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.007579 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.008232 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.008322 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:20.008388 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:20.009623 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0811 07:41:20.009705 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:41:20.010402 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:20.002865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:39101 (local address 127.18.72.193:51869)
0811 07:41:20.003692 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:20.003707 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:20.003833 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:20.004223 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:20.004227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:20.004976 (+ 749us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:20.005139 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:20.005148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.006164 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.006167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:20.007440 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:20.007450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.008523 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.008526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:20.008628 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:20.008650 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:41:20.010178 (+ 1528us) client_negotiation.cc:240] Negotiation successful
0811 07:41:20.010252 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":117,"threads_started":1}
I20260811 07:41:20.011093 686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68d5e052eea479ca6095198a9a73fc2" candidate_uuid: "fe0af2c0de58472eb25a601e29afc340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69819e4ca77347c7afce90c7e3205323" is_pre_election: true
I20260811 07:41:20.011358 686 raft_consensus.cc:2454] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe0af2c0de58472eb25a601e29afc340 in term 0.
I20260811 07:41:20.011924 328 leader_election.cc:302] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69819e4ca77347c7afce90c7e3205323, fe0af2c0de58472eb25a601e29afc340; no voters:
I20260811 07:41:20.012199 779 raft_consensus.cc:2781] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:41:20.012257 779 raft_consensus.cc:482] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:41:20.012295 779 raft_consensus.cc:3037] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:20.013284 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:20.003494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:36083 (local address 127.18.72.194:44513)
0811 07:41:20.003642 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:20.003647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:20.004229 (+ 582us) server_negotiation.cc:407] Connection header received
0811 07:41:20.006343 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:20.006347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:20.006396 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:20.006443 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:20.008447 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.009223 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.010871 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.011521 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.011614 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:20.011660 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:20.013070 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0811 07:41:20.013153 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:41:20.013684 781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:20.003051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:44513 (local address 127.18.72.193:36083)
0811 07:41:20.004123 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:20.004135 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:20.006145 (+ 2010us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x1e14f6e8. stack: 00007f5d6f5cfc78 00007f5d6f5d01a9 00007f5d6f1921d7 00000000005984a6 00000000005984f9 000000000059c488 00007f5d6f5cc2ab 00007f5d6f5cb9da 00007f5d77d29817 00007f5d77d3c07f 00007f5d77d32003 00007f5d73c24267 00007f5d73c22819 00007f5d73c63832 00007f5d73c6468c 00007f5d73c7a0d3
0811 07:41:20.006174 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:20.006576 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:20.006578 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:20.006595 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:20.006725 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:20.006730 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.009463 (+ 2733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.009467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:20.010721 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:20.010733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.011992 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.011995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:20.012100 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:20.012120 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 07:41:20.013472 (+ 1352us) client_negotiation.cc:240] Negotiation successful
0811 07:41:20.013545 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"spinlock_wait_cycles":2900864,"thread_start_us":109,"threads_started":1}
I20260811 07:41:20.013648 779 raft_consensus.cc:504] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
I20260811 07:41:20.014067 779 leader_election.cc:288] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [CANDIDATE]: Term 1 election: Requested vote from peers 69819e4ca77347c7afce90c7e3205323 (127.18.72.195:39101), 95fcf803849144e49a662597c54226db (127.18.72.194:44513)
I20260811 07:41:20.014285 520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68d5e052eea479ca6095198a9a73fc2" candidate_uuid: "fe0af2c0de58472eb25a601e29afc340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fcf803849144e49a662597c54226db" is_pre_election: true
I20260811 07:41:20.014495 533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68d5e052eea479ca6095198a9a73fc2" candidate_uuid: "fe0af2c0de58472eb25a601e29afc340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95fcf803849144e49a662597c54226db"
I20260811 07:41:20.014577 520 raft_consensus.cc:2454] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe0af2c0de58472eb25a601e29afc340 in term 0.
I20260811 07:41:20.015789 686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68d5e052eea479ca6095198a9a73fc2" candidate_uuid: "fe0af2c0de58472eb25a601e29afc340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69819e4ca77347c7afce90c7e3205323"
I20260811 07:41:20.015950 686 raft_consensus.cc:3037] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:20.016880 686 raft_consensus.cc:2454] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe0af2c0de58472eb25a601e29afc340 in term 1.
I20260811 07:41:20.017197 328 leader_election.cc:302] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69819e4ca77347c7afce90c7e3205323, fe0af2c0de58472eb25a601e29afc340; no voters: 95fcf803849144e49a662597c54226db
I20260811 07:41:20.017495 779 raft_consensus.cc:2781] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:20.017705 779 raft_consensus.cc:686] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [term 1 LEADER]: Becoming Leader. State: Replica: fe0af2c0de58472eb25a601e29afc340, State: Running, Role: LEADER
I20260811 07:41:20.018041 779 consensus_queue.cc:237] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } }
I20260811 07:41:20.021363 32702 catalog_manager.cc:5495] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe0af2c0de58472eb25a601e29afc340 (127.18.72.193). New cstate: current_term: 1 leader_uuid: "fe0af2c0de58472eb25a601e29afc340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe0af2c0de58472eb25a601e29afc340" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 45577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 } health_report { overall_health: UNKNOWN } } }
I20260811 07:41:20.056483 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:20.051439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.18.72.193:45577)
0811 07:41:20.051563 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:20.051568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:20.051582 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:41:20.051701 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:20.051704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:20.051778 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:20.051926 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:41:20.052173 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.052976 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.054271 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:20.054632 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:20.054733 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:20.055152 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:41:20.055722 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:41:20.055844 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:20.056257 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 07:41:20.056326 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:41:20.063753 533 raft_consensus.cc:3037] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:20.064010 686 raft_consensus.cc:1261] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 1 FOLLOWER]: Refusing update from remote peer fe0af2c0de58472eb25a601e29afc340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:20.064755 783 consensus_queue.cc:1035] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69819e4ca77347c7afce90c7e3205323" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:20.065454 533 raft_consensus.cc:1261] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [term 1 FOLLOWER]: Refusing update from remote peer fe0af2c0de58472eb25a601e29afc340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:20.071131 779 consensus_queue.cc:1035] T b68d5e052eea479ca6095198a9a73fc2 P fe0af2c0de58472eb25a601e29afc340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95fcf803849144e49a662597c54226db" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:20.079308 786 mvcc.cc:205] Tried to move back new op lower bound from 7317233991932895232 to 7317233991756783616. Current Snapshot: MvccSnapshot[applied={T|T < 7317233991932895232}]
I20260811 07:41:20.087644 791 mvcc.cc:205] Tried to move back new op lower bound from 7317233991932895232 to 7317233991756783616. Current Snapshot: MvccSnapshot[applied={T|T < 7317233991932895232}]
I20260811 07:41:20.091190 32703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:41:20.091358 32703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:41:20.094216 32703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bcbfcc5f52b40e28f43cc2f8169d560: Sending DeleteTablet for 3 replicas of tablet b68d5e052eea479ca6095198a9a73fc2
I20260811 07:41:20.095009 665 tablet_service.cc:1514] Processing DeleteTablet for tablet b68d5e052eea479ca6095198a9a73fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:20 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:57626
I20260811 07:41:20.095304 357 tablet_service.cc:1514] Processing DeleteTablet for tablet b68d5e052eea479ca6095198a9a73fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:20 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:55588
I20260811 07:41:20.095813 501 tablet_service.cc:1514] Processing DeleteTablet for tablet b68d5e052eea479ca6095198a9a73fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:20 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:38106
I20260811 07:41:20.097123 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 32746
I20260811 07:41:20.100103 799 tablet_replica.cc:330] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: stopping tablet replica
I20260811 07:41:20.100502 799 raft_consensus.cc:2229] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:41:20.100872 799 raft_consensus.cc:2258] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:20.101289 797 tablet_replica.cc:330] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: stopping tablet replica
I20260811 07:41:20.102887 799 ts_tablet_manager.cc:1905] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:41:20.105016 797 raft_consensus.cc:2229] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:41:20.105453 797 raft_consensus.cc:2258] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:20.107434 799 ts_tablet_manager.cc:1918] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:20.107611 799 log.cc:1196] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/b68d5e052eea479ca6095198a9a73fc2
I20260811 07:41:20.108125 799 ts_tablet_manager.cc:1939] T b68d5e052eea479ca6095198a9a73fc2 P 95fcf803849144e49a662597c54226db: Deleting consensus metadata
I20260811 07:41:20.109786 32689 catalog_manager.cc:4841] TS 95fcf803849144e49a662597c54226db (127.18.72.194:44513): tablet b68d5e052eea479ca6095198a9a73fc2 (table test-table [id=cd98583b701f42e99dcca51de73941ce]) successfully deleted
I20260811 07:41:20.111368 797 ts_tablet_manager.cc:1905] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 07:41:20.114564 32689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:45577 (error 108)
I20260811 07:41:20.114787 797 ts_tablet_manager.cc:1918] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:20.114885 797 log.cc:1196] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/b68d5e052eea479ca6095198a9a73fc2
I20260811 07:41:20.115235 797 ts_tablet_manager.cc:1939] T b68d5e052eea479ca6095198a9a73fc2 P 69819e4ca77347c7afce90c7e3205323: Deleting consensus metadata
I20260811 07:41:20.115684 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 453
I20260811 07:41:20.117771 32688 catalog_manager.cc:4841] TS 69819e4ca77347c7afce90c7e3205323 (127.18.72.195:39101): tablet b68d5e052eea479ca6095198a9a73fc2 (table test-table [id=cd98583b701f42e99dcca51de73941ce]) successfully deleted
I20260811 07:41:20.120047 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:20.119667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:45577 (local address 127.0.0.1:55608)
0811 07:41:20.119811 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:20.119941 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:45577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 07:41:20.120217 32689 catalog_manager.cc:4584] TS fe0af2c0de58472eb25a601e29afc340 (127.18.72.193:45577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b68d5e052eea479ca6095198a9a73fc2: Network error: Client connection negotiation failed: client connection to 127.18.72.193:45577: connect: Connection refused (error 111)
I20260811 07:41:20.127667 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 595
I20260811 07:41:20.137465 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 32669
I20260811 07:41:20.146737 18723 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 07:41:20.490823 18723 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 07:41:20.490962 18723 mini_ranger.cc:66] Stopping Ranger...
I20260811 07:41:20.835291 18723 mini_ranger.cc:68] Stopped Ranger
2026-08-11 07:41:20.835 UTC [31677] LOG: received smart shutdown request
2026-08-11 07:41:20.839 UTC [31677] LOG: background worker "logical replication launcher" (PID 31686) exited with exit code 1
2026-08-11 07:41:20.840 UTC [31681] LOG: shutting down
2026-08-11 07:41:20.991 UTC [31677] LOG: database system is shut down
2026-08-11T07:41:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48631 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:41:21 dist-test-slave-zkpd krb5kdc[810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:41:21 dist-test-slave-zkpd krb5kdc[810](info): set up 2 sockets
Aug 11 07:41:21 dist-test-slave-zkpd krb5kdc[810](info): commencing operation
krb5kdc: starting...
W20260811 07:41:22.212801 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:41:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:41:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:41:22.334653 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:42125
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:42125,127.18.72.253:37873,127.18.72.252:46037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:22.447748 823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:22.448050 823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:22.448130 823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:22.452067 823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:22.452147 823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:22.452183 823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:22.452245 823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:22.452279 823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:22.457324 823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:42125,127.18.72.253:37873,127.18.72.252:46037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:42125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:22.458418 823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:22.459421 823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:22.467716 831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:22.467831 823 server_base.cc:1034] running on GCE node
W20260811 07:41:22.467728 828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:22.467809 829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:22.468590 823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:22.469645 823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:22.470845 823 hybrid_clock.cc:648] HybridClock initialized: now 1786434082470814 us; error 43 us; skew 500 ppm
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:22.473627 823 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:41:22.474471 823 webserver.cc:460] Webserver started at http://127.18.72.254:37529/ using document root <none> and password file <none>
I20260811 07:41:22.475014 823 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:22.475075 823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:22.475315 823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:22.477306 823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "5cbcdb575e454e1ca90addf05fe93dad"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "71043664fa792e344f3ada25bd930a1f"
server_key_iv: "0439ee86f0ee89eb9ebcb1e882011397"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.477869 823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "5cbcdb575e454e1ca90addf05fe93dad"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "71043664fa792e344f3ada25bd930a1f"
server_key_iv: "0439ee86f0ee89eb9ebcb1e882011397"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.481474 823 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:41:22.483685 838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:22.484822 823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:41:22.484944 823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "5cbcdb575e454e1ca90addf05fe93dad"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "71043664fa792e344f3ada25bd930a1f"
server_key_iv: "0439ee86f0ee89eb9ebcb1e882011397"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.485038 823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:22.506908 823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:22.510154 823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:22.510370 823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:22.517395 823 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:42125
I20260811 07:41:22.517407 896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:42125 every 8 connection(s)
I20260811 07:41:22.518287 823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:41:22.520588 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 823
I20260811 07:41:22.520714 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:41:22.521018 18723 external_mini_cluster.cc:1428] Setting key 5b2e1c4ed053041e6510f00f97b92035
I20260811 07:41:22.521098 897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:22.526207 897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.72.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.253@KRBTEST.COM" created.
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:22.542321 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.529006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42125 (local address 127.0.0.1:39902)
0811 07:41:22.529478 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.529531 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.530348 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.531161 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.531166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.531485 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.531834 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.531857 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.533028 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.533031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.533921 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.533931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.534607 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.534611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.534723 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.535440 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.535462 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.537342 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.539269 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.539279 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.539290 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.539520 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.539845 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.539849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.539852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.540003 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.540597 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.540603 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.540746 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.540919 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.541323 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":106,"thread_start_us":150,"threads_started":1}
I20260811 07:41:22.542321 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.529011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.18.72.254:42125)
0811 07:41:22.529550 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.529562 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.529638 (+ 76us) server_negotiation.cc:407] Connection header received
0811 07:41:22.530678 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.530684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.530954 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.531162 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.532042 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.532904 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.534042 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.534428 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.534601 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.537480 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.537485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.537496 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.537518 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.539139 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.539631 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.539637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.539639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.539745 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.540103 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.540106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.540108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.540476 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.540614 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.540999 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.541243 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"spinlock_wait_cycles":12160,"thread_start_us":145,"threads_started":1}
I20260811 07:41:22.544950 897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:41:22.545740 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.545506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45540)
0811 07:41:22.545615 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.545682 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 07:41:22.546028 842 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
I20260811 07:41:22.546914 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.546723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45542)
0811 07:41:22.546821 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.546865 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 07:41:22.547120 897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:37873: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab.
Entry for principal kudu/127.18.72.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab.
I20260811 07:41:22.549081 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.253:37873
--webserver_interface=127.18.72.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab
--principal=kudu/127.18.72.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:42125,127.18.72.253:37873,127.18.72.252:46037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:41:22.596364 897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } attempt: 1
I20260811 07:41:22.597208 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.596923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45552)
0811 07:41:22.597068 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.597133 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 07:41:22.598047 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.597801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45554)
0811 07:41:22.597939 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.597990 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 07:41:22.598234 897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:37873: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
W20260811 07:41:22.660563 904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:22.660802 904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:22.660841 904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:22.664603 904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:22.664645 904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:22.664664 904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:22.664680 904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:22.664695 904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:22.666412 897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } attempt: 2
I20260811 07:41:22.667223 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.666920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45556)
0811 07:41:22.667065 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.667151 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 07:41:22.668001 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.667740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45566)
0811 07:41:22.667846 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.667940 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 07:41:22.668151 897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:37873: Network error: Client connection negotiation failed: client connection to 127.18.72.253:37873: connect: Connection refused (error 111)
I20260811 07:41:22.670011 904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:42125,127.18.72.253:37873,127.18.72.252:46037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab
--principal=kudu/127.18.72.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.253:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.72.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:22.670950 904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:22.671959 904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:22.680179 912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:22.680145 910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:22.680357 904 server_base.cc:1034] running on GCE node
W20260811 07:41:22.680145 909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:22.681079 904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:22.682107 904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:22.683264 904 hybrid_clock.cc:648] HybridClock initialized: now 1786434082683232 us; error 45 us; skew 500 ppm
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:22.686179 904 init.cc:377] Logged in from keytab as kudu/127.18.72.253@KRBTEST.COM (short username kudu)
I20260811 07:41:22.687100 904 webserver.cc:460] Webserver started at http://127.18.72.253:37991/ using document root <none> and password file <none>
I20260811 07:41:22.687660 904 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:22.687752 904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:22.688050 904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:22.690656 904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/data/instance:
uuid: "a6788c80ee9542148d4849ba4455105c"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "a57afb846a9b57ca83e440ee29bc1f88"
server_key_iv: "3c72f00e61fb03b7780e9db4217a2387"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.691300 904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/wal/instance:
uuid: "a6788c80ee9542148d4849ba4455105c"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "a57afb846a9b57ca83e440ee29bc1f88"
server_key_iv: "3c72f00e61fb03b7780e9db4217a2387"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.695048 904 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:41:22.697563 920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:22.698791 904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:41:22.698944 904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/wal
uuid: "a6788c80ee9542148d4849ba4455105c"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "a57afb846a9b57ca83e440ee29bc1f88"
server_key_iv: "3c72f00e61fb03b7780e9db4217a2387"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.699045 904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:22.716514 904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:22.719702 904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:22.719956 904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:22.727180 904 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.253:37873
I20260811 07:41:22.727195 975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.253:37873 every 8 connection(s)
I20260811 07:41:22.728204 904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
I20260811 07:41:22.730954 976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:22.734771 976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:41:22.735275 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 904
I20260811 07:41:22.735419 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-1/wal/instance
I20260811 07:41:22.735694 18723 external_mini_cluster.cc:1428] Setting key 8f50d1ae40b17de0a9ce6ac4039635a2
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.252@KRBTEST.COM" created.
I20260811 07:41:22.749382 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.736821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.18.72.254:42125)
0811 07:41:22.736987 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.736993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.737882 (+ 889us) server_negotiation.cc:407] Connection header received
0811 07:41:22.738813 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.738817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.738884 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.738989 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.740181 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.740715 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.742366 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.742608 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.742701 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.745528 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.745531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.745534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.745560 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.747046 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.747568 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.747573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.747574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.747659 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.748083 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.748088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.748089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.748351 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.748460 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.748968 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.749118 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:41:22.749941 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.737159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42125 (local address 127.0.0.1:39912)
0811 07:41:22.737691 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.737750 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.738602 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.739159 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.739165 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.739502 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.740036 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.740053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.740852 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.740862 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.742228 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.742245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.742774 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.742780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.743009 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.743688 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.743710 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.745362 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.747187 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.747193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.747207 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.747464 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.747753 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.747757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.747759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.747985 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.748471 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.748475 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.748700 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.748856 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.749084 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260811 07:41:22.751355 976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
I20260811 07:41:22.759294 897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } attempt: 3
I20260811 07:41:22.759685 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.751804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45572)
0811 07:41:22.751932 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.751948 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.752020 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.752542 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.752544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.752719 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.752799 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.752804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.753740 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.753743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.754259 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.754266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.754839 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.754842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.754965 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.755402 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.755416 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.756622 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.758201 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.758207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.758209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.758394 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.758675 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.758680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.758682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.758762 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.759205 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.759208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.759292 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.759417 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.759537 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":3}
I20260811 07:41:22.759950 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.751960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45572 (local address 127.18.72.253:37873)
0811 07:41:22.752222 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.752231 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.752248 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:41:22.752278 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.752281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.752429 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.752563 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.752902 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.753605 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.754379 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.754724 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.754841 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.756754 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.756764 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.756781 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.756802 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.758090 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.758500 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.758505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.758507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.758585 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.758862 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.758865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.758866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.759096 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.759226 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.759568 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.759745 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260811 07:41:22.761786 976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:41:22.762428 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.762166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:43932)
0811 07:41:22.762269 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.762350 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab.
Entry for principal kudu/127.18.72.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab.
W20260811 07:41:22.762969 924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
I20260811 07:41:22.763316 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.252:46037
--webserver_interface=127.18.72.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab
--principal=kudu/127.18.72.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:42125,127.18.72.253:37873,127.18.72.252:46037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:41:22.763725 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.763530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:43944)
0811 07:41:22.763628 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.763673 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 07:41:22.763859 976 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46037: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
I20260811 07:41:22.769645 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.759715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45580)
0811 07:41:22.759879 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.759925 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.760005 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.760373 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.760377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.760541 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.760692 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.760705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.761638 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.761642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.762310 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.762320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.764190 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.764196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.764336 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.764737 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.764757 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.766319 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.768157 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.768161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.768163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.768379 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.768692 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.768695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.768697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.768781 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.769140 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.769145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.769240 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.769358 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.769486 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 07:41:22.769658 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.759768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.18.72.253:37873)
0811 07:41:22.760110 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.760120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.760134 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:41:22.760168 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.760170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.760219 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.760356 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.760814 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.761525 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.762417 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.764054 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.764148 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.766476 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.766479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.766481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.766500 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.768048 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.768513 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.768518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.768520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.768599 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.768873 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.768876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.768877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.769036 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.769134 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.769412 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.769516 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":143,"threads_started":1}
I20260811 07:41:22.770519 897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:41:22.771220 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.771003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:43946)
0811 07:41:22.771110 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.771157 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 07:41:22.772074 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.771734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:43954)
0811 07:41:22.771975 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.772024 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260811 07:41:22.772249 897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46037: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
I20260811 07:41:22.803407 897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } attempt: 1
I20260811 07:41:22.804280 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.804004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:43960)
0811 07:41:22.804149 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.804209 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 07:41:22.805472 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.805244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:43976)
0811 07:41:22.805360 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.805416 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 07:41:22.805598 897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46037: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
I20260811 07:41:22.813154 976 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } attempt: 1
I20260811 07:41:22.813890 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.813628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:43990)
0811 07:41:22.813767 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.813822 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 07:41:22.814584 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.814375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:43998)
0811 07:41:22.814484 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.814534 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 07:41:22.814707 976 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46037: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
W20260811 07:41:22.878539 985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:22.878824 985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:22.878897 985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 07:41:22.880749 897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } attempt: 2
I20260811 07:41:22.881551 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.881266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:44006)
0811 07:41:22.881408 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.881477 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 07:41:22.882248 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.882040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:44022)
0811 07:41:22.882146 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.882199 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 07:41:22.882367 897 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46037: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
I20260811 07:41:22.882861 976 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } attempt: 2
W20260811 07:41:22.883134 985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:22.883205 985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:22.883250 985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:22.883296 985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:22.883335 985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:22.883558 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.883276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:44034)
0811 07:41:22.883420 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.883482 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 07:41:22.884284 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.884063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:44048)
0811 07:41:22.884172 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.884224 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 07:41:22.884418 976 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46037: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46037: connect: Connection refused (error 111)
I20260811 07:41:22.888870 985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:42125,127.18.72.253:37873,127.18.72.252:46037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab
--principal=kudu/127.18.72.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.252:46037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.72.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:22.889859 985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:22.890854 985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:22.899034 990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:22.899048 993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:22.899067 991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:22.899324 985 server_base.cc:1034] running on GCE node
I20260811 07:41:22.900013 985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:22.901106 985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:22.902304 985 hybrid_clock.cc:648] HybridClock initialized: now 1786434082902282 us; error 37 us; skew 500 ppm
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:22.905123 985 init.cc:377] Logged in from keytab as kudu/127.18.72.252@KRBTEST.COM (short username kudu)
I20260811 07:41:22.905957 985 webserver.cc:460] Webserver started at http://127.18.72.252:43283/ using document root <none> and password file <none>
I20260811 07:41:22.906464 985 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:22.906522 985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:22.906677 985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:22.908648 985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/data/instance:
uuid: "65f38368a35a4494891674bbcde7c361"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "37c53e1cde2814ccfa765b069b545091"
server_key_iv: "ba8c4791d0b045169e1468d6596a082a"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.909147 985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/wal/instance:
uuid: "65f38368a35a4494891674bbcde7c361"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "37c53e1cde2814ccfa765b069b545091"
server_key_iv: "ba8c4791d0b045169e1468d6596a082a"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.912683 985 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:41:22.914958 1001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:22.916188 985 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:41:22.916297 985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/wal
uuid: "65f38368a35a4494891674bbcde7c361"
format_stamp: "Formatted at 2026-08-11 07:41:22 on dist-test-slave-zkpd"
server_key: "37c53e1cde2814ccfa765b069b545091"
server_key_iv: "ba8c4791d0b045169e1468d6596a082a"
server_key_version: "encryptionkey@0"
I20260811 07:41:22.916378 985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:22.934674 985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:22.937708 985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:22.937856 985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:22.944932 985 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.252:46037
I20260811 07:41:22.944944 1056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.252:46037 every 8 connection(s)
I20260811 07:41:22.945900 985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
I20260811 07:41:22.948731 1057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:22.950206 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 985
I20260811 07:41:22.950330 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786433920963264-18723-0/minicluster-data/master-2/wal/instance
I20260811 07:41:22.950651 18723 external_mini_cluster.cc:1428] Setting key 1def1436f4023ee6d05c712cb17e7abb
I20260811 07:41:22.953078 1057 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:22.964552 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.954843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.18.72.254:42125)
0811 07:41:22.954970 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.954976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.955239 (+ 263us) server_negotiation.cc:407] Connection header received
0811 07:41:22.955407 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.955410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.955463 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.955548 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.956258 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.956755 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.957683 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.957907 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.958015 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.960759 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.960765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.960771 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.960814 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.962381 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.962889 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.962893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.962895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.962998 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.963398 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.963404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.963407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.963609 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.963710 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.964263 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.964380 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:41:22.965262 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.952092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.18.72.252:46037)
0811 07:41:22.952499 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.952521 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.952555 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:41:22.953206 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.953223 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.953591 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.953947 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.954756 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.956006 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.956852 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.957321 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.957499 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.960190 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.960198 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.960216 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.960238 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.962301 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.962789 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.962795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.962797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.962964 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.963339 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.963342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.963343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.963835 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.964041 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.964320 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.964659 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":196,"server-negotiator.queue_time_us":274,"thread_start_us":134,"threads_started":1}
I20260811 07:41:22.965261 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.954789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42125 (local address 127.0.0.1:39926)
0811 07:41:22.955125 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.955162 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.955301 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.955602 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.955608 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.955992 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.956134 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.956140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.956882 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.956890 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.957561 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.957570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.958053 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.958057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.958154 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.958668 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.958683 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.960617 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.962482 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.962491 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.962502 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.962793 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.963116 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.963119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.963121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.963302 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.963712 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.963717 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.963939 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.964136 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.964470 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":71,"thread_start_us":97,"threads_started":1}
I20260811 07:41:22.966575 1057 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:22.975234 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.967011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:37873 (local address 127.0.0.1:45594)
0811 07:41:22.967288 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.967299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.967361 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.967754 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.967762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.967940 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.968042 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.968050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.968898 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.968901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.969537 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.969547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.970008 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.970013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.970093 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.970531 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.970542 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.971856 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.973719 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.973722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.973723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.974014 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.974287 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.974293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.974296 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.974404 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.974753 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.974756 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.974862 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.974969 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.975091 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260811 07:41:22.975343 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.967141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45594 (local address 127.18.72.253:37873)
0811 07:41:22.967339 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.967345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.967365 (+ 20us) server_negotiation.cc:407] Connection header received
0811 07:41:22.967433 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.967436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.967490 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.967573 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.968160 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.968775 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.969652 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.969894 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.969970 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.972031 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.972035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.972037 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.972062 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.973589 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.974116 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.974118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.974119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.974193 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.974499 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.974502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.974503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.974658 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.974730 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.975054 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.975182 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 07:41:22.975566 976 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } attempt: 3
I20260811 07:41:22.976293 1057 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:41:22.976778 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.967670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39942 (local address 127.18.72.254:42125)
0811 07:41:22.967773 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.967776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.967786 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:41:22.967818 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.967820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.967849 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.967953 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.968559 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.969126 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.970049 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.970274 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.970355 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.972983 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.972986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.972989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.973012 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.974541 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.975012 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.975015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.975017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.975095 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.975396 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.975400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.975401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.975642 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.975908 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.976497 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.976622 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
I20260811 07:41:22.981498 897 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } attempt: 3
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
I20260811 07:41:22.986862 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.976720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:44080)
0811 07:41:22.977010 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.977024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.977428 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.977710 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.977713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.977865 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.977930 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.977935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.978773 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.978779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.979512 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.979522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.980151 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.980155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.980237 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.980720 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.980737 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.982831 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.985108 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.985113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.985118 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.985518 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.985832 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.985838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.985840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.985930 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.986387 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.986389 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.986478 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.986577 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.986704 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"spinlock_wait_cycles":2944}
Aug 11 07:41:22 dist-test-slave-zkpd krb5kdc[810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434082, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
I20260811 07:41:22.987500 978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.975955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:44074)
0811 07:41:22.976074 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.976088 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.976163 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.976421 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.976424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.976611 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.976706 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.976713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.978158 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.978165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.979050 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.979066 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.979526 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.979530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.979615 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.980097 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.980118 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.982412 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.985901 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.985904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.985906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.986211 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.986512 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.986515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.986518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.986607 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.987037 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.987040 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.987130 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.987214 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.987341 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":3}
I20260811 07:41:22.987521 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.976863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.18.72.252:46037)
0811 07:41:22.977177 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.977186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.977199 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:41:22.977532 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.977535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.977570 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.977634 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.978027 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.978671 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.979628 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.980042 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.980149 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.982964 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.982969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.982972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.982996 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.984993 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.985623 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.985626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.985629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.985731 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.986031 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.986035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.986038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.986237 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.986353 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.987263 (+ 910us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.987376 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260811 07:41:22.988368 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.976080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44074 (local address 127.18.72.252:46037)
0811 07:41:22.976193 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.976196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.976211 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:41:22.976242 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.976245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.976292 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.976821 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.976920 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.978074 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.979162 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.979409 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.979513 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.982555 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.982559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.982561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.982585 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.985518 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.986322 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.986328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.986330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.986404 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.986720 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.986723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.986724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.986911 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.988059 (+ 1148us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.988136 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.988243 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51,"spinlock_wait_cycles":2816}
I20260811 07:41:22.988780 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.977931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45598 (local address 127.18.72.253:37873)
0811 07:41:22.978027 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.978031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.978085 (+ 54us) server_negotiation.cc:407] Connection header received
0811 07:41:22.978220 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.978223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.978264 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.978336 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.979017 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.979745 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.980832 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.981161 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.981277 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.983961 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.983967 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.983970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.983996 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.986412 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.986900 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.986905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.986907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.986994 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.987314 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.987319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.987320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.987536 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.987748 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.988436 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.988605 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":38}
I20260811 07:41:22.991303 1057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361: Bootstrap starting.
I20260811 07:41:22.991488 976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c: Bootstrap starting.
I20260811 07:41:22.992255 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:41:22.993974 976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:22.994129 1057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:22.994940 976 log.cc:824] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:22.994946 1057 log.cc:824] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:22.998281 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.981847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46037 (local address 127.0.0.1:44086)
0811 07:41:22.981963 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:22.981977 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:22.982050 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:22.982597 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:22.982600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:22.982770 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:22.982901 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.982912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.984009 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.984013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.984789 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:22.984801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.985271 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.985275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:22.985386 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.985842 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:22.985861 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:22.987226 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:22.996692 (+ 9466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.996698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.996702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.996959 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.997273 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:22.997275 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:22.997277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:22.997373 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:22.997816 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:22.997819 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:22.997911 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 07:41:22.998012 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 07:41:22.998139 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 07:41:22.998344 1057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361: No bootstrap required, opened a new log
I20260811 07:41:22.998744 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.981992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.18.72.252:46037)
0811 07:41:22.982305 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.982316 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.982331 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:41:22.982369 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.982372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.982422 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.983020 (+ 598us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.983151 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.983929 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.984924 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.985160 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.985258 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:22.987494 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:22.987498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:22.987500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:22.987522 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:22.996574 (+ 9052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.997073 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.997076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.997078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.997158 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:22.997474 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:22.997477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:22.997478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:22.997682 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:22.997771 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:22.998489 (+ 718us) server_negotiation.cc:299] Negotiation successful
0811 07:41:22.998591 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":115,"threads_started":1}
I20260811 07:41:23.002928 1057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.003222 1057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:23.003288 1057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f38368a35a4494891674bbcde7c361, State: Initialized, Role: FOLLOWER
I20260811 07:41:23.003767 1057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.005292 1057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:23.005538 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.995249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.18.72.254:42125)
0811 07:41:22.995362 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.995367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.995382 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:41:22.995417 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.995422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.995465 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.995542 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.996502 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.997275 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.998592 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.998862 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:22.998961 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:23.000528 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:23.000531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:23.000534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:23.000557 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:23.002567 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:23.003048 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:23.003055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:23.003058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:23.003148 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:23.003600 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:23.003605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:23.003609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:23.003839 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:23.004446 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:23.005224 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 07:41:23.005347 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:41:23.007422 897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad: Bootstrap starting.
I20260811 07:41:23.008674 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.995998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44100 (local address 127.18.72.252:46037)
0811 07:41:22.996125 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.996131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.999121 (+ 2990us) server_negotiation.cc:407] Connection header received
0811 07:41:22.999217 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.999220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.999271 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.999623 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:23.000123 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:23.000833 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:23.001885 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:23.002254 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:23.002345 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:23.004207 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:23.004213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:23.004219 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:23.004245 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:23.006556 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:23.007105 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:23.007108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:23.007110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:23.007223 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:23.007635 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:23.007640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:23.007642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:23.007884 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:23.008031 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:23.008407 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 07:41:23.008515 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:41:23.010852 897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad: Neither blocks nor log segments found. Creating new log.
W20260811 07:41:23.011082 1084 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:41:23.011166 1084 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:41:23.011361 1073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.011482 1073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:41:23.011814 897 log.cc:824] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:23.009635 985 master_runner.cc:186] Error getting master registration for 127.18.72.254:42125: OK, instance_id { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" instance_seqno: 1786434082515224 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:41:23.012596 985 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:41:23.013518 976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c: No bootstrap required, opened a new log
I20260811 07:41:23.013950 897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad: No bootstrap required, opened a new log
I20260811 07:41:23.016920 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:22.995359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.18.72.253:37873)
0811 07:41:22.995448 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:22.995453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:22.995469 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:41:22.995503 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:22.995507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:22.995555 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:22.995630 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:22.996621 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:22.997334 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:23.000845 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:23.001147 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:23.001219 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:23.013263 (+ 12044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:23.013267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:23.013270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:23.013297 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:23.015026 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:23.015506 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:23.015509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:23.015511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:23.015610 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:23.015938 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:23.015942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:23.015943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:23.016168 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:23.016278 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:23.016637 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 07:41:23.016763 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 07:41:23.018100 897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.018322 897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:23.018388 897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbcdb575e454e1ca90addf05fe93dad, State: Initialized, Role: FOLLOWER
I20260811 07:41:23.018850 897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.018774 976 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.019016 976 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:23.019078 976 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6788c80ee9542148d4849ba4455105c, State: Initialized, Role: FOLLOWER
I20260811 07:41:23.019565 976 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.020160 1087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.020306 897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:23.020512 1088 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.020651 1088 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:41:23.020947 976 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:23.022790 1087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:41:23.025599 1103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:41:23.025679 1103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:41:23.030189 904 master_runner.cc:263] No leader master found from master a6788c80ee9542148d4849ba4455105c
I20260811 07:41:23.030261 904 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:41:23.030285 823 master_runner.cc:263] No leader master found from master 5cbcdb575e454e1ca90addf05fe93dad
I20260811 07:41:23.030364 823 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 07:41:23.030758 1111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:41:23.030844 1111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:41:23.035054 1087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:23.035255 1087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.036692 948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5cbcdb575e454e1ca90addf05fe93dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6788c80ee9542148d4849ba4455105c" is_pre_election: true
I20260811 07:41:23.036820 1087 leader_election.cc:288] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6788c80ee9542148d4849ba4455105c (127.18.72.253:37873), 65f38368a35a4494891674bbcde7c361 (127.18.72.252:46037)
I20260811 07:41:23.037016 948 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbcdb575e454e1ca90addf05fe93dad in term 0.
I20260811 07:41:23.037015 1029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5cbcdb575e454e1ca90addf05fe93dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65f38368a35a4494891674bbcde7c361" is_pre_election: true
I20260811 07:41:23.037290 1029 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cbcdb575e454e1ca90addf05fe93dad in term 0.
I20260811 07:41:23.037829 842 leader_election.cc:302] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbcdb575e454e1ca90addf05fe93dad, a6788c80ee9542148d4849ba4455105c; no voters:
I20260811 07:41:23.038218 1087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:41:23.038305 1087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:41:23.038360 1087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:23.039386 1087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.039942 1087 leader_election.cc:288] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [CANDIDATE]: Term 1 election: Requested vote from peers a6788c80ee9542148d4849ba4455105c (127.18.72.253:37873), 65f38368a35a4494891674bbcde7c361 (127.18.72.252:46037)
I20260811 07:41:23.039986 1029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5cbcdb575e454e1ca90addf05fe93dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65f38368a35a4494891674bbcde7c361"
I20260811 07:41:23.039980 948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5cbcdb575e454e1ca90addf05fe93dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6788c80ee9542148d4849ba4455105c"
I20260811 07:41:23.040093 1029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:23.040102 948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:23.041355 948 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbcdb575e454e1ca90addf05fe93dad in term 1.
I20260811 07:41:23.041728 842 leader_election.cc:302] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbcdb575e454e1ca90addf05fe93dad, a6788c80ee9542148d4849ba4455105c; no voters:
I20260811 07:41:23.041749 1029 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cbcdb575e454e1ca90addf05fe93dad in term 1.
I20260811 07:41:23.041946 1087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:23.042210 1087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [term 1 LEADER]: Becoming Leader. State: Replica: 5cbcdb575e454e1ca90addf05fe93dad, State: Running, Role: LEADER
I20260811 07:41:23.042575 1087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } }
I20260811 07:41:23.044456 1087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cbcdb575e454e1ca90addf05fe93dad. Latest consensus state: current_term: 1 leader_uuid: "5cbcdb575e454e1ca90addf05fe93dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.044662 1087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:23.049485 1117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:41:23.050450 1117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:41:23.057576 1029 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [term 1 FOLLOWER]: Refusing update from remote peer 5cbcdb575e454e1ca90addf05fe93dad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:23.057662 948 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [term 1 FOLLOWER]: Refusing update from remote peer 5cbcdb575e454e1ca90addf05fe93dad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:23.058223 1114 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [LEADER]: Connected to new peer: Peer: permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:23.058396 1087 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:23.061977 1073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cbcdb575e454e1ca90addf05fe93dad. Latest consensus state: current_term: 1 leader_uuid: "5cbcdb575e454e1ca90addf05fe93dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.062091 1073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:41:23.063441 1088 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cbcdb575e454e1ca90addf05fe93dad. Latest consensus state: current_term: 1 leader_uuid: "5cbcdb575e454e1ca90addf05fe93dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.063592 1088 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:41:23.067826 1087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5cbcdb575e454e1ca90addf05fe93dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.067966 1087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:23.069212 1088 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5cbcdb575e454e1ca90addf05fe93dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.069353 1088 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:41:23.069639 1118 mvcc.cc:205] Tried to move back new op lower bound from 7317234004195962880 to 7317234004145782784. Current Snapshot: MvccSnapshot[applied={T|T < 7317234004195962880}]
I20260811 07:41:23.069850 1120 mvcc.cc:205] Tried to move back new op lower bound from 7317234004195962880 to 7317234004145782784. Current Snapshot: MvccSnapshot[applied={T|T < 7317234004195962880}]
I20260811 07:41:23.071907 1117 catalog_manager.cc:1347] Generated new cluster ID: d690354537b44d0ea34fa5053f51c20c
I20260811 07:41:23.071964 1117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:41:23.075103 1114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5cbcdb575e454e1ca90addf05fe93dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.075225 1114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:23.076256 1073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5cbcdb575e454e1ca90addf05fe93dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbcdb575e454e1ca90addf05fe93dad" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42125 } } peers { permanent_uuid: "a6788c80ee9542148d4849ba4455105c" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 37873 } } peers { permanent_uuid: "65f38368a35a4494891674bbcde7c361" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46037 } } }
I20260811 07:41:23.076366 1073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:41:23.077042 1119 mvcc.cc:205] Tried to move back new op lower bound from 7317234004195962880 to 7317234004145782784. Current Snapshot: MvccSnapshot[applied={T|T < 7317234004195962880}]
I20260811 07:41:23.092694 1117 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:41:23.094022 1117 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:41:23.117041 1117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cbcdb575e454e1ca90addf05fe93dad: Generated new TSK 0
I20260811 07:41:23.118027 1117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:41:24.012513 1084 catalog_manager.cc:1259] Loaded cluster ID: d690354537b44d0ea34fa5053f51c20c
I20260811 07:41:24.012624 1084 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361: loading cluster ID for follower catalog manager: success
I20260811 07:41:24.014319 1084 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361: acquiring CA information for follower catalog manager: success
I20260811 07:41:24.015529 1084 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65f38368a35a4494891674bbcde7c361: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:41:24.032817 1111 catalog_manager.cc:1259] Loaded cluster ID: d690354537b44d0ea34fa5053f51c20c
I20260811 07:41:24.032888 1111 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c: loading cluster ID for follower catalog manager: success
I20260811 07:41:24.034566 1111 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c: acquiring CA information for follower catalog manager: success
I20260811 07:41:24.035750 1111 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6788c80ee9542148d4849ba4455105c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:41:24.203827 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 823
I20260811 07:41:24.213824 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 904
I20260811 07:41:24.221426 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 985
2026-08-11T07:41:24Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3107 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:41:24 dist-test-slave-zkpd krb5kdc[1133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:41:24 dist-test-slave-zkpd krb5kdc[1133](info): set up 2 sockets
Aug 11 07:41:24 dist-test-slave-zkpd krb5kdc[1133](info): commencing operation
krb5kdc: starting...
W20260811 07:41:25.310206 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:41:25 dist-test-slave-zkpd krb5kdc[1133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:41:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:41:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:41:25.431820 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45643
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:45907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:45643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:25.544580 1148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:25.544839 1148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:25.544878 1148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:25.548709 1148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:25.548753 1148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:25.548771 1148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:25.548789 1148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:25.548802 1148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:25.553900 1148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:45907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:45643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:25.554782 1148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:25.555704 1148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:25.563912 1155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:25.563932 1158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:25.564059 1148 server_base.cc:1034] running on GCE node
W20260811 07:41:25.564230 1156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:25.564746 1148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:25.565790 1148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:25.566954 1148 hybrid_clock.cc:648] HybridClock initialized: now 1786434085566928 us; error 39 us; skew 500 ppm
Aug 11 07:41:25 dist-test-slave-zkpd krb5kdc[1133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434085, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:25.569572 1148 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:41:25.570434 1148 webserver.cc:460] Webserver started at http://127.18.72.254:44081/ using document root <none> and password file <none>
I20260811 07:41:25.570979 1148 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:25.571070 1148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:25.571296 1148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:25.573285 1148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d"
format_stamp: "Formatted at 2026-08-11 07:41:25 on dist-test-slave-zkpd"
server_key: "3a2c8e55b12fc0bec16661b810efd7a2"
server_key_iv: "56e754bbb7fe3f6b8e06c97b6826ada7"
server_key_version: "encryptionkey@0"
I20260811 07:41:25.573845 1148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d"
format_stamp: "Formatted at 2026-08-11 07:41:25 on dist-test-slave-zkpd"
server_key: "3a2c8e55b12fc0bec16661b810efd7a2"
server_key_iv: "56e754bbb7fe3f6b8e06c97b6826ada7"
server_key_version: "encryptionkey@0"
I20260811 07:41:25.577433 1148 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:41:25.579722 1165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:25.581027 1148 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:41:25.581178 1148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d"
format_stamp: "Formatted at 2026-08-11 07:41:25 on dist-test-slave-zkpd"
server_key: "3a2c8e55b12fc0bec16661b810efd7a2"
server_key_iv: "56e754bbb7fe3f6b8e06c97b6826ada7"
server_key_version: "encryptionkey@0"
I20260811 07:41:25.581290 1148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:25.594576 1148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:25.597252 1148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:25.597457 1148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:25.604472 1148 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:45643
I20260811 07:41:25.604483 1217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:45643 every 8 connection(s)
I20260811 07:41:25.605362 1148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:41:25.607371 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 1148
I20260811 07:41:25.607506 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:41:25.607834 18723 external_mini_cluster.cc:1428] Setting key 1006a47f9b05ea94eb4c4b923ac5fd88
I20260811 07:41:25.608457 1218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:25.614008 1218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d: Bootstrap starting.
Aug 11 07:41:25 dist-test-slave-zkpd krb5kdc[1133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:25.616760 1218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:25.617611 1218 log.cc:824] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:25.619884 1218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d: No bootstrap required, opened a new log
I20260811 07:41:25.621778 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:25.609109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.18.72.254:45643)
0811 07:41:25.609546 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:25.609561 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:25.609594 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:41:25.610212 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:25.610228 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:25.610537 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:25.610859 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:25.611676 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:25.612713 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:25.613560 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:25.614036 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:25.614214 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:25.617058 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:25.617064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:25.617077 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:25.617104 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:25.618941 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:25.619422 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:25.619427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:25.619432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:25.619536 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:25.619845 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:25.619847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:25.619848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:25.620324 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:25.620467 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:25.620771 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 07:41:25.621071 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":145,"threads_started":1}
I20260811 07:41:25.623970 1218 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45643 } }
I20260811 07:41:25.624204 1218 raft_consensus.cc:374] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:25.624280 1218 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69ea97f973e4c3ab7e1cab74ad64c3d, State: Initialized, Role: FOLLOWER
I20260811 07:41:25.624719 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:41:25.624751 1218 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45643 } }
I20260811 07:41:25.624939 1218 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:41:25.625007 1218 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:41:25.625113 1218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:25.626075 1218 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45643 } }
I20260811 07:41:25.626453 1218 leader_election.cc:302] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69ea97f973e4c3ab7e1cab74ad64c3d; no voters:
I20260811 07:41:25.626767 1218 leader_election.cc:288] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:41:25.626919 1223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:25.627159 1223 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [term 1 LEADER]: Becoming Leader. State: Replica: c69ea97f973e4c3ab7e1cab74ad64c3d, State: Running, Role: LEADER
I20260811 07:41:25.627470 1223 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45643 } }
I20260811 07:41:25.628002 1218 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:25.629148 1224 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45643 } } }
I20260811 07:41:25.629189 1225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69ea97f973e4c3ab7e1cab74ad64c3d. Latest consensus state: current_term: 1 leader_uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69ea97f973e4c3ab7e1cab74ad64c3d" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45643 } } }
I20260811 07:41:25.629277 1224 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:25.629297 1225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:25.629627 1232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:41:25.632642 1232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:41:25.638250 1232 catalog_manager.cc:1347] Generated new cluster ID: fe968bbdf69a49008c7f7d0308e8038f
I20260811 07:41:25.638320 1232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:41:25.649708 1232 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:41:25.650568 1232 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:41:25.655297 1232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c69ea97f973e4c3ab7e1cab74ad64c3d: Generated new TSK 0
I20260811 07:41:25.655910 1232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:41:25.680389 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 1148
2026-08-11T07:41:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1457 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:41:25 dist-test-slave-zkpd krb5kdc[1246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:41:25 dist-test-slave-zkpd krb5kdc[1246](info): set up 2 sockets
Aug 11 07:41:25 dist-test-slave-zkpd krb5kdc[1246](info): commencing operation
krb5kdc: starting...
W20260811 07:41:26.739605 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:41:26 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:41:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:41:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:41:26.859170 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:37847
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:37847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:26.968415 1260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:26.968650 1260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:26.968688 1260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:26.972514 1260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:26.972560 1260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:26.972579 1260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:26.972595 1260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:26.972609 1260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:26.977685 1260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:37847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:37847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:26.978588 1260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:26.979512 1260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:26.987807 1266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:26.987840 1265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:26.987777 1268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:26.988092 1260 server_base.cc:1034] running on GCE node
I20260811 07:41:26.988658 1260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:26.989677 1260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:26.990841 1260 hybrid_clock.cc:648] HybridClock initialized: now 1786434086990806 us; error 43 us; skew 500 ppm
Aug 11 07:41:26 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434086, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:26.997570 1260 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:41:26.998490 1260 webserver.cc:460] Webserver started at http://127.18.72.254:37851/ using document root <none> and password file <none>
I20260811 07:41:26.999063 1260 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:26.999154 1260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:26.999384 1260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:27.001438 1260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "b3494d0d27f2400d8c74102a5ee60635"
format_stamp: "Formatted at 2026-08-11 07:41:26 on dist-test-slave-zkpd"
server_key: "fa4a3582046cae680844e1fa94a17ff0"
server_key_iv: "b19572a21e841f2f6058f7e071935940"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.002012 1260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "b3494d0d27f2400d8c74102a5ee60635"
format_stamp: "Formatted at 2026-08-11 07:41:26 on dist-test-slave-zkpd"
server_key: "fa4a3582046cae680844e1fa94a17ff0"
server_key_iv: "b19572a21e841f2f6058f7e071935940"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.005748 1260 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 07:41:27.008122 1276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.009295 1260 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:41:27.009436 1260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "b3494d0d27f2400d8c74102a5ee60635"
format_stamp: "Formatted at 2026-08-11 07:41:26 on dist-test-slave-zkpd"
server_key: "fa4a3582046cae680844e1fa94a17ff0"
server_key_iv: "b19572a21e841f2f6058f7e071935940"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.009543 1260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:27.030010 1260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:27.033231 1260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:27.033428 1260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:27.040445 1260 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:37847
I20260811 07:41:27.040753 1335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:37847 every 8 connection(s)
I20260811 07:41:27.041448 1260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:41:27.044345 1336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:27.045181 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 1260
I20260811 07:41:27.045339 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:41:27.045621 18723 external_mini_cluster.cc:1428] Setting key d0601fa82e468442226ecbd0be8b55da
I20260811 07:41:27.050926 1336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Bootstrap starting.
I20260811 07:41:27.053555 1336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:27.054349 1336 log.cc:824] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Log is configured to *not* fsync() on all Append() calls
Aug 11 07:41:27 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:27.056458 1336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: No bootstrap required, opened a new log
I20260811 07:41:27.060081 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.047004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.18.72.254:37847)
0811 07:41:27.047421 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.047441 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.047486 (+ 45us) server_negotiation.cc:407] Connection header received
0811 07:41:27.048364 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.048387 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.048780 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.049146 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:27.049813 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.051223 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.052104 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.052759 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.052911 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.055597 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:27.055602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:27.055614 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:27.055637 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:27.057307 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.057792 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.057796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.057800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.057915 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.058226 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.058228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.058229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.058586 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:27.058736 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.059037 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.059340 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":98,"threads_started":1}
I20260811 07:41:27.060461 1336 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:27.060693 1336 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:27.060767 1336 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3494d0d27f2400d8c74102a5ee60635, State: Initialized, Role: FOLLOWER
I20260811 07:41:27.061154 1336 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:27.061316 1336 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:41:27.061404 1336 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:41:27.061524 1336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:27.062366 1336 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:27.062692 1336 leader_election.cc:302] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3494d0d27f2400d8c74102a5ee60635; no voters:
I20260811 07:41:27.062968 1336 leader_election.cc:288] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:41:27.063069 1341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:27.063299 1341 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 1 LEADER]: Becoming Leader. State: Replica: b3494d0d27f2400d8c74102a5ee60635, State: Running, Role: LEADER
I20260811 07:41:27.063669 1341 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:27.064179 1336 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:27.065495 1343 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3494d0d27f2400d8c74102a5ee60635. Latest consensus state: current_term: 1 leader_uuid: "b3494d0d27f2400d8c74102a5ee60635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } } }
I20260811 07:41:27.065511 1342 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3494d0d27f2400d8c74102a5ee60635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } } }
I20260811 07:41:27.065639 1342 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:27.065639 1343 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:27.066094 1350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:41:27.068799 1350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:41:27.073760 1350 catalog_manager.cc:1347] Generated new cluster ID: bcab5ec127e047299487ce1c9de1be82
I20260811 07:41:27.073820 1350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:41:27.086308 1350 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:41:27.087515 1350 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:41:27.091909 1350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Generated new TSK 0
I20260811 07:41:27.092622 1350 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:41:27.126439 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37847
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:27.238471 1362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:27.238699 1362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:27.238740 1362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:27.242630 1362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:27.242672 1362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:27.242750 1362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:41:27.247633 1362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:27.248610 1362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:27.249588 1362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:27.258957 1368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:27.259019 1370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:27.258973 1367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:27.259223 1362 server_base.cc:1034] running on GCE node
I20260811 07:41:27.259675 1362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:27.260317 1362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:27.261480 1362 hybrid_clock.cc:648] HybridClock initialized: now 1786434087261467 us; error 23 us; skew 500 ppm
Aug 11 07:41:27 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434087, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:27.264240 1362 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:41:27.265208 1362 webserver.cc:460] Webserver started at http://127.18.72.193:43449/ using document root <none> and password file <none>
I20260811 07:41:27.265770 1362 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:27.265828 1362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:27.266063 1362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:27.268112 1362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "79693eda0b5a4c20acf9150cf322f939"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "ca84fa8d58cdbecc670d2540f39ca0be"
server_key_iv: "b3d5908c7f1977e933fe340ce49afb15"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.268613 1362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "79693eda0b5a4c20acf9150cf322f939"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "ca84fa8d58cdbecc670d2540f39ca0be"
server_key_iv: "b3d5908c7f1977e933fe340ce49afb15"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.272451 1362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 07:41:27.274803 1378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.276309 1362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:41:27.276551 1362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "79693eda0b5a4c20acf9150cf322f939"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "ca84fa8d58cdbecc670d2540f39ca0be"
server_key_iv: "b3d5908c7f1977e933fe340ce49afb15"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.276656 1362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:27.286895 1362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:27.289880 1362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:27.290094 1362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:27.290704 1362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:27.291553 1362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:27.291600 1362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.291663 1362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:27.291695 1362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.300647 1362 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:39709
I20260811 07:41:27.300675 1504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:39709 every 8 connection(s)
I20260811 07:41:27.301501 1362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:41:27.302546 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 1362
I20260811 07:41:27.302695 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:41:27.303078 18723 external_mini_cluster.cc:1428] Setting key e0aed0a772e794e64d270f6ad9b68a94
Aug 11 07:41:27 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434087, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
I20260811 07:41:27.317190 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.303484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:36685 (local address 127.18.72.254:37847)
0811 07:41:27.303833 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.303838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.304727 (+ 889us) server_negotiation.cc:407] Connection header received
0811 07:41:27.306560 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.306566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.306662 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.306811 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:27.308722 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.309268 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.310526 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.310760 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.310874 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.313359 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:27.313363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:27.313366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:27.313391 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:27.315003 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.315466 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.315469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.315470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.315570 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.315882 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.315884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.315885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.316123 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:27.316235 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.316783 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.316924 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260811 07:41:27.317728 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.303965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:36685)
0811 07:41:27.304574 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.304624 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:27.306244 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:27.307242 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:27.307253 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:27.307829 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:27.308566 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.308585 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.309401 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.309404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.310399 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.310409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.310906 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.310910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.311080 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.311656 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:27.311671 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:27.313202 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:27.315108 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:27.315112 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:27.315123 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:27.315373 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:27.315661 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:27.315667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:27.315668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:27.315796 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:27.316234 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:27.316243 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:27.316540 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 07:41:27.316684 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 07:41:27.316922 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":140,"threads_started":1}
I20260811 07:41:27.319002 1505 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37847
I20260811 07:41:27.319253 1505 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:27.319804 1505 heartbeater.cc:511] Master 127.18.72.254:37847 requested a full tablet report, sending...
I20260811 07:41:27.321331 1296 ts_manager.cc:194] Registered new tserver with Master: 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709)
I20260811 07:41:27.322422 1296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:36685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:41:27.332757 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37847
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:27.444154 1510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:27.444428 1510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:27.444505 1510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:27.448426 1510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:27.448508 1510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:27.448597 1510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:41:27.453670 1510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:27.454676 1510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:27.455651 1510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:27.465294 1515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:27.465442 1518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:27.465301 1516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:27.466419 1510 server_base.cc:1034] running on GCE node
I20260811 07:41:27.466882 1510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:27.467617 1510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:27.468786 1510 hybrid_clock.cc:648] HybridClock initialized: now 1786434087468746 us; error 49 us; skew 500 ppm
Aug 11 07:41:27 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434087, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:27.471328 1510 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:41:27.472193 1510 webserver.cc:460] Webserver started at http://127.18.72.194:35645/ using document root <none> and password file <none>
I20260811 07:41:27.472752 1510 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:27.472841 1510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:27.473078 1510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:27.475046 1510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "7e479c50b025432bb94681789b3eb54a"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "37000ef11f5ac9b161f9ef6d523c7aff"
server_key_iv: "bb7df9e5930a2f726e2b40e316a26888"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.475589 1510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "7e479c50b025432bb94681789b3eb54a"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "37000ef11f5ac9b161f9ef6d523c7aff"
server_key_iv: "bb7df9e5930a2f726e2b40e316a26888"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.479091 1510 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:41:27.481523 1525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.482695 1510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 07:41:27.482827 1510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "7e479c50b025432bb94681789b3eb54a"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "37000ef11f5ac9b161f9ef6d523c7aff"
server_key_iv: "bb7df9e5930a2f726e2b40e316a26888"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.482934 1510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:27.496572 1510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:27.499155 1510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:27.499347 1510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:27.499876 1510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:27.500718 1510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:27.500788 1510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.500854 1510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:27.500907 1510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.509821 1510 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:42025
I20260811 07:41:27.509862 1650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:42025 every 8 connection(s)
I20260811 07:41:27.510651 1510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:41:27.518797 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 1510
I20260811 07:41:27.518937 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:41:27.519284 18723 external_mini_cluster.cc:1428] Setting key 1d2a24db3570e39b4bd3c547781650d5
Aug 11 07:41:27 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434087, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:27.524607 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.512460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:45543 (local address 127.18.72.254:37847)
0811 07:41:27.512588 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.512592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.513422 (+ 830us) server_negotiation.cc:407] Connection header received
0811 07:41:27.514910 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.514913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.514977 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.515075 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:27.516243 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.516771 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.517900 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.518130 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.518237 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.521085 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:27.521089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:27.521092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:27.521125 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:27.522483 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.522994 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.522997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.522999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.523084 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.523445 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.523450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.523452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.523633 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:27.523741 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.524317 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.524440 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:41:27.525352 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.512781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:45543)
0811 07:41:27.513272 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.513324 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:27.514687 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:27.515240 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:27.515246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:27.515588 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:27.516104 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.516118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.516911 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.516915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.517770 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.517780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.518296 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.518300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.518458 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.519221 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:27.519242 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:27.520919 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:27.522604 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:27.522610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:27.522624 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:27.522889 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:27.523187 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:27.523190 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:27.523192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:27.523347 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:27.523741 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:27.523749 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:27.524040 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 07:41:27.524257 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 07:41:27.524565 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":119,"threads_started":1}
I20260811 07:41:27.526508 1651 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37847
I20260811 07:41:27.526741 1651 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:27.527199 1651 heartbeater.cc:511] Master 127.18.72.254:37847 requested a full tablet report, sending...
I20260811 07:41:27.528267 1296 ts_manager.cc:194] Registered new tserver with Master: 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025)
I20260811 07:41:27.528803 1296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:45543
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:41:27.546636 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37847
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:27.660017 1656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:27.660244 1656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:27.660290 1656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:27.664228 1656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:27.664309 1656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:27.664381 1656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:41:27.669734 1656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:27.670650 1656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:27.671543 1656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:27.681166 1661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:27.681182 1664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:27.681293 1656 server_base.cc:1034] running on GCE node
W20260811 07:41:27.681211 1662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:27.681885 1656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:27.682468 1656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:27.683653 1656 hybrid_clock.cc:648] HybridClock initialized: now 1786434087683644 us; error 29 us; skew 500 ppm
Aug 11 07:41:27 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434087, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:27.686338 1656 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:41:27.687189 1656 webserver.cc:460] Webserver started at http://127.18.72.195:44809/ using document root <none> and password file <none>
I20260811 07:41:27.687713 1656 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:27.687772 1656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:27.688001 1656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:27.689949 1656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "66711bb2dc1144cfbf72b2d12d181c91"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "16d00f7eb694f651e1f21772059c3538"
server_key_iv: "66d00abddc017dc9664f464c3efd1bac"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.690455 1656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "66711bb2dc1144cfbf72b2d12d181c91"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "16d00f7eb694f651e1f21772059c3538"
server_key_iv: "66d00abddc017dc9664f464c3efd1bac"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.693981 1656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:41:27.696256 1671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.697400 1656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:41:27.697490 1656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "66711bb2dc1144cfbf72b2d12d181c91"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "16d00f7eb694f651e1f21772059c3538"
server_key_iv: "66d00abddc017dc9664f464c3efd1bac"
server_key_version: "encryptionkey@0"
I20260811 07:41:27.697569 1656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:27.709240 1656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:27.711788 1656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:27.712000 1656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:27.712606 1656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:27.713402 1656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:27.713447 1656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.713514 1656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:27.713549 1656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:27.722210 1656 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:40959
I20260811 07:41:27.722230 1794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:40959 every 8 connection(s)
I20260811 07:41:27.723109 1656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
Aug 11 07:41:27 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434087, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:27.732664 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 1656
I20260811 07:41:27.732803 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:41:27.733086 18723 external_mini_cluster.cc:1428] Setting key 3cfa25549cbedc7bcbd83d582fb61f12
I20260811 07:41:27.737226 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.724960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:49035 (local address 127.18.72.254:37847)
0811 07:41:27.725112 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.725116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.725938 (+ 822us) server_negotiation.cc:407] Connection header received
0811 07:41:27.727365 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.727368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.727428 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.727535 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:27.728777 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.729313 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.730462 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.730680 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.730793 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.733313 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:27.733317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:27.733319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:27.733346 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:27.735227 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.735722 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.735726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.735728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.735815 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.736171 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.736175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.736177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.736365 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:27.736480 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.736945 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.737063 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:41:27.737941 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.725347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:49035)
0811 07:41:27.725785 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.725829 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:27.727128 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:27.727717 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:27.727724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:27.728140 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:27.728621 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.728635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.729442 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.729450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.730339 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.730348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.730845 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.730849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.731021 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.731603 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:27.731619 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:27.733122 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:27.735350 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:27.735358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:27.735374 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:27.735616 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:27.735926 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:27.735932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:27.735936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:27.736072 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:27.736480 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:27.736488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:27.736709 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 07:41:27.736886 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 07:41:27.737187 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":117,"threads_started":1}
I20260811 07:41:27.738966 1795 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37847
I20260811 07:41:27.739198 1795 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:27.739658 1795 heartbeater.cc:511] Master 127.18.72.254:37847 requested a full tablet report, sending...
I20260811 07:41:27.740674 1296 ts_manager.cc:194] Registered new tserver with Master: 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959)
I20260811 07:41:27.741215 1296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:49035
I20260811 07:41:27.746899 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:41:27.759734 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.748828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.18.72.254:37847)
0811 07:41:27.748920 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.748923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.749158 (+ 235us) server_negotiation.cc:407] Connection header received
0811 07:41:27.749242 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.749244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.749278 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.749343 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:27.750008 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.750482 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.751227 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.751426 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.751498 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.756889 (+ 5391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:27.756892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:27.756893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:27.756915 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:27.758412 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.758755 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.758758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.758759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.758839 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:27.759057 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:27.759059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:27.759060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:27.759253 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:27.759356 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.759541 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.759629 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 07:41:27.762552 1296 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:41:27.765722 1296 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:41:27.780088 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.773807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:42025 (local address 127.0.0.1:43694)
0811 07:41:27.774633 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.774674 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:27.774798 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:27.775824 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:27.775827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:27.775845 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:27.776131 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.776138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.777484 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.777490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.778664 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.778674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.779503 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.779507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.779613 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.779663 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 07:41:27.779773 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 07:41:27.779917 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":23,"thread_start_us":97,"threads_started":1}
I20260811 07:41:27.780205 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.775078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60792 (local address 127.18.72.195:40959)
0811 07:41:27.775493 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.775504 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.775527 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:41:27.775604 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.775614 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.775822 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.776005 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:27.776331 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.777040 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.778495 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.779277 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.779399 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.779544 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.779975 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.780069 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":102,"threads_started":1}
I20260811 07:41:27.780443 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.773848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.18.72.193:39709)
0811 07:41:27.774209 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.774221 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.774766 (+ 545us) server_negotiation.cc:407] Connection header received
0811 07:41:27.775005 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.775012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.775231 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.775410 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:27.775688 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.777067 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.778734 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.779519 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.779620 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.779748 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.780042 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.780150 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":177,"threads_started":1}
I20260811 07:41:27.780874 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.774471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.18.72.194:42025)
0811 07:41:27.775383 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.775397 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.775420 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:41:27.775492 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.775497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.775676 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.775831 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:27.776262 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.777370 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.778770 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.779380 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.779482 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.779620 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.780494 (+ 874us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.780729 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":119,"threads_started":1}
I20260811 07:41:27.780145 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.774420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:40959 (local address 127.0.0.1:60792)
0811 07:41:27.774885 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.774899 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:27.774984 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:27.776039 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:27.776042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:27.776059 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:27.776183 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.776192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.777168 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.777174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.778371 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.778381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.779611 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.779615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.779701 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.779792 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 07:41:27.779935 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 07:41:27.780014 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":39,"thread_start_us":93,"threads_started":1}
I20260811 07:41:27.780879 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.773669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39709 (local address 127.0.0.1:50980)
0811 07:41:27.774624 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.774674 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:27.774826 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:27.775404 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:27.775411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:27.775437 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:27.775561 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.775567 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.777212 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.777215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.778602 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.778614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.779699 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.779702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.779856 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.779880 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 07:41:27.780684 (+ 804us) client_negotiation.cc:240] Negotiation successful
0811 07:41:27.780752 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":171,"thread_start_us":70,"threads_started":1}
I20260811 07:41:27.783798 1579 tablet_service.cc:1467] Processing CreateTablet for tablet 2182831c14fb463cad1520c15d8c6deb (DEFAULT_TABLE table=test-table [id=c786517bf290425aae599d0d9dc6ea10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:27.784247 1723 tablet_service.cc:1467] Processing CreateTablet for tablet 2182831c14fb463cad1520c15d8c6deb (DEFAULT_TABLE table=test-table [id=c786517bf290425aae599d0d9dc6ea10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:27.784910 1579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2182831c14fb463cad1520c15d8c6deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:27.785391 1723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2182831c14fb463cad1520c15d8c6deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:27.785915 1431 tablet_service.cc:1467] Processing CreateTablet for tablet 2182831c14fb463cad1520c15d8c6deb (DEFAULT_TABLE table=test-table [id=c786517bf290425aae599d0d9dc6ea10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:27.786931 1431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2182831c14fb463cad1520c15d8c6deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:27.791261 1812 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: Bootstrap starting.
I20260811 07:41:27.792768 1813 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: Bootstrap starting.
I20260811 07:41:27.794121 1814 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Bootstrap starting.
I20260811 07:41:27.794728 1812 tablet_bootstrap.cc:654] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:27.794910 1813 tablet_bootstrap.cc:654] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:27.795710 1812 log.cc:824] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:27.796052 1814 tablet_bootstrap.cc:654] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:27.796197 1813 log.cc:824] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:27.797363 1814 log.cc:824] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:27.798061 1813 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: No bootstrap required, opened a new log
I20260811 07:41:27.798061 1812 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: No bootstrap required, opened a new log
I20260811 07:41:27.798254 1812 ts_tablet_manager.cc:1397] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 07:41:27.798254 1813 ts_tablet_manager.cc:1397] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 07:41:27.799028 1814 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: No bootstrap required, opened a new log
I20260811 07:41:27.799225 1814 ts_tablet_manager.cc:1397] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 07:41:27.802253 1813 raft_consensus.cc:348] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.802253 1812 raft_consensus.cc:348] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.802505 1812 raft_consensus.cc:374] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:27.802557 1812 raft_consensus.cc:729] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e479c50b025432bb94681789b3eb54a, State: Initialized, Role: FOLLOWER
I20260811 07:41:27.802431 1814 raft_consensus.cc:348] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.802626 1813 raft_consensus.cc:374] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:27.802670 1814 raft_consensus.cc:374] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:27.802681 1813 raft_consensus.cc:729] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66711bb2dc1144cfbf72b2d12d181c91, State: Initialized, Role: FOLLOWER
I20260811 07:41:27.802726 1814 raft_consensus.cc:729] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79693eda0b5a4c20acf9150cf322f939, State: Initialized, Role: FOLLOWER
I20260811 07:41:27.803036 1812 consensus_queue.cc:260] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.803036 1813 consensus_queue.cc:260] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.803165 1814 consensus_queue.cc:260] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.803766 1505 heartbeater.cc:503] Master 127.18.72.254:37847 was elected leader, sending a full tablet report...
I20260811 07:41:27.803966 1813 ts_tablet_manager.cc:1428] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 07:41:27.803966 1812 ts_tablet_manager.cc:1428] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 07:41:27.804049 1814 ts_tablet_manager.cc:1428] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 07:41:27.804302 1795 heartbeater.cc:503] Master 127.18.72.254:37847 was elected leader, sending a full tablet report...
I20260811 07:41:27.804294 1651 heartbeater.cc:503] Master 127.18.72.254:37847 was elected leader, sending a full tablet report...
I20260811 07:41:27.831382 1819 raft_consensus.cc:482] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:27.831605 1819 raft_consensus.cc:504] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.832787 1819 leader_election.cc:288] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709), 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025)
I20260811 07:41:27.836511 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.832979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39709 (local address 127.18.72.195:43483)
0811 07:41:27.833105 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.833123 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:27.833211 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:27.833519 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:27.833522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:27.833543 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:27.833675 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.833686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.834497 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.834500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.835336 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.835346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.836072 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.836076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.836192 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.836226 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 07:41:27.836299 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:41:27.836374 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 07:41:27.836552 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.833136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:43483 (local address 127.18.72.193:39709)
0811 07:41:27.833269 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.833273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.833301 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:41:27.833355 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.833358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.833408 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.833491 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:27.833806 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.834391 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.835464 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.835964 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.836058 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.836098 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.836341 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.836413 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:41:27.836724 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.833161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:43719 (local address 127.18.72.194:42025)
0811 07:41:27.833359 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.833363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.833492 (+ 129us) server_negotiation.cc:407] Connection header received
0811 07:41:27.833582 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.833585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.833634 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.833734 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:27.834020 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.834523 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.835587 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.836146 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.836254 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.836315 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.836524 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.836601 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 07:41:27.836876 1821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.833025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:42025 (local address 127.18.72.195:43719)
0811 07:41:27.833387 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.833401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:27.833503 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:27.833768 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:27.833771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:27.833805 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:27.833913 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.833920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.834631 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.834634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.835469 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:27.835480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.836266 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.836288 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:27.836393 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.836416 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 07:41:27.836493 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 07:41:27.836553 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":1,"thread_start_us":123,"threads_started":1}
I20260811 07:41:27.837138 1454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2182831c14fb463cad1520c15d8c6deb" candidate_uuid: "66711bb2dc1144cfbf72b2d12d181c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79693eda0b5a4c20acf9150cf322f939" is_pre_election: true
I20260811 07:41:27.837427 1454 raft_consensus.cc:2454] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66711bb2dc1144cfbf72b2d12d181c91 in term 0.
I20260811 07:41:27.837481 1600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2182831c14fb463cad1520c15d8c6deb" candidate_uuid: "66711bb2dc1144cfbf72b2d12d181c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e479c50b025432bb94681789b3eb54a" is_pre_election: true
I20260811 07:41:27.837810 1600 raft_consensus.cc:2454] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66711bb2dc1144cfbf72b2d12d181c91 in term 0.
I20260811 07:41:27.837915 1672 leader_election.cc:302] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66711bb2dc1144cfbf72b2d12d181c91, 79693eda0b5a4c20acf9150cf322f939; no voters:
I20260811 07:41:27.838183 1819 raft_consensus.cc:2781] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:41:27.838270 1819 raft_consensus.cc:482] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:41:27.838325 1819 raft_consensus.cc:3037] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:27.839340 1819 raft_consensus.cc:504] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.839733 1819 leader_election.cc:288] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [CANDIDATE]: Term 1 election: Requested vote from peers 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709), 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025)
I20260811 07:41:27.840072 1454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2182831c14fb463cad1520c15d8c6deb" candidate_uuid: "66711bb2dc1144cfbf72b2d12d181c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79693eda0b5a4c20acf9150cf322f939"
I20260811 07:41:27.840188 1600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2182831c14fb463cad1520c15d8c6deb" candidate_uuid: "66711bb2dc1144cfbf72b2d12d181c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e479c50b025432bb94681789b3eb54a"
I20260811 07:41:27.840214 1454 raft_consensus.cc:3037] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:27.840330 1600 raft_consensus.cc:3037] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:27.841266 1454 raft_consensus.cc:2454] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66711bb2dc1144cfbf72b2d12d181c91 in term 1.
I20260811 07:41:27.841369 1600 raft_consensus.cc:2454] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66711bb2dc1144cfbf72b2d12d181c91 in term 1.
I20260811 07:41:27.841547 1672 leader_election.cc:302] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66711bb2dc1144cfbf72b2d12d181c91, 79693eda0b5a4c20acf9150cf322f939; no voters:
I20260811 07:41:27.841758 1819 raft_consensus.cc:2781] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:27.841951 1819 raft_consensus.cc:686] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 LEADER]: Becoming Leader. State: Replica: 66711bb2dc1144cfbf72b2d12d181c91, State: Running, Role: LEADER
I20260811 07:41:27.842296 1819 consensus_queue.cc:237] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:27.845305 1294 catalog_manager.cc:5495] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195). New cstate: current_term: 1 leader_uuid: "66711bb2dc1144cfbf72b2d12d181c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } health_report { overall_health: HEALTHY } } }
I20260811 07:41:27.863433 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.859968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.18.72.195:40959)
0811 07:41:27.860092 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:27.860099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:27.860118 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:41:27.860302 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:27.860310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:27.860360 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:27.860467 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:41:27.860703 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.861198 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.862050 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:27.862261 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:27.862367 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:27.862663 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:41:27.863054 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:41:27.863190 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:27.863232 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 07:41:27.863282 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:41:27.869148 1600 raft_consensus.cc:1261] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 1 FOLLOWER]: Refusing update from remote peer 66711bb2dc1144cfbf72b2d12d181c91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:27.869148 1454 raft_consensus.cc:1261] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Refusing update from remote peer 66711bb2dc1144cfbf72b2d12d181c91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:27.869823 1819 consensus_queue.cc:1035] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:27.870004 1822 consensus_queue.cc:1035] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:27.879158 1827 mvcc.cc:205] Tried to move back new op lower bound from 7317234023905312768 to 7317234023804891136. Current Snapshot: MvccSnapshot[applied={T|T < 7317234023905312768 or (T in {7317234023905312768})}]
I20260811 07:41:27.886364 1295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:41:27.886533 1295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:41:27.888944 1295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Sending DeleteTablet for 3 replicas of tablet 2182831c14fb463cad1520c15d8c6deb
I20260811 07:41:27.889681 1579 tablet_service.cc:1514] Processing DeleteTablet for tablet 2182831c14fb463cad1520c15d8c6deb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:27 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43694
I20260811 07:41:27.889739 1723 tablet_service.cc:1514] Processing DeleteTablet for tablet 2182831c14fb463cad1520c15d8c6deb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:27 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:60792
I20260811 07:41:27.890201 1834 tablet_replica.cc:330] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: stopping tablet replica
I20260811 07:41:27.890199 1431 tablet_service.cc:1514] Processing DeleteTablet for tablet 2182831c14fb463cad1520c15d8c6deb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:27 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:50980
I20260811 07:41:27.890532 1834 raft_consensus.cc:2229] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:41:27.890703 1836 tablet_replica.cc:330] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: stopping tablet replica
I20260811 07:41:27.890698 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 1362
I20260811 07:41:27.890848 1834 raft_consensus.cc:2258] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:27.891101 1836 raft_consensus.cc:2229] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:41:27.891194 1835 tablet_replica.cc:330] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: stopping tablet replica
I20260811 07:41:27.891443 1835 raft_consensus.cc:2229] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:41:27.891738 1835 raft_consensus.cc:2258] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:27.893412 1835 ts_tablet_manager.cc:1905] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:41:27.896643 1834 ts_tablet_manager.cc:1905] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:41:27.897608 1835 ts_tablet_manager.cc:1918] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:27.897814 1835 log.cc:1196] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/2182831c14fb463cad1520c15d8c6deb
I20260811 07:41:27.898283 1835 ts_tablet_manager.cc:1939] T 2182831c14fb463cad1520c15d8c6deb P 66711bb2dc1144cfbf72b2d12d181c91: Deleting consensus metadata
I20260811 07:41:27.899310 1834 ts_tablet_manager.cc:1918] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:27.899421 1834 log.cc:1196] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/2182831c14fb463cad1520c15d8c6deb
I20260811 07:41:27.899761 1834 ts_tablet_manager.cc:1939] T 2182831c14fb463cad1520c15d8c6deb P 7e479c50b025432bb94681789b3eb54a: Deleting consensus metadata
I20260811 07:41:27.899859 1277 catalog_manager.cc:4841] TS 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959): tablet 2182831c14fb463cad1520c15d8c6deb (table test-table [id=c786517bf290425aae599d0d9dc6ea10]) successfully deleted
I20260811 07:41:27.901000 1280 catalog_manager.cc:4841] TS 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025): tablet 2182831c14fb463cad1520c15d8c6deb (table test-table [id=c786517bf290425aae599d0d9dc6ea10]) successfully deleted
W20260811 07:41:27.908282 1277 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:39709 (error 108)
I20260811 07:41:27.908563 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 1510
I20260811 07:41:27.916807 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:27.916476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39709 (local address 127.0.0.1:50990)
0811 07:41:27.916630 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:27.916714 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 07:41:27.917066 1277 catalog_manager.cc:4584] TS 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2182831c14fb463cad1520c15d8c6deb: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39709: connect: Connection refused (error 111)
I20260811 07:41:27.919328 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 1656
I20260811 07:41:27.928583 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 1260
I20260811 07:41:27.946612 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:37847
--webserver_interface=127.18.72.254
--webserver_port=37851
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:37847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:28.059753 1838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:28.060101 1838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:28.060187 1838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:28.064108 1838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:28.064181 1838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:28.064218 1838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:28.064253 1838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:28.064287 1838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:28.069775 1838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:37847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:37847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=37851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:28.070685 1838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:28.071635 1838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:28.084887 1846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:28.084883 1848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:28.084924 1845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:28.085084 1838 server_base.cc:1034] running on GCE node
I20260811 07:41:28.085752 1838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:28.086841 1838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:28.088028 1838 hybrid_clock.cc:648] HybridClock initialized: now 1786434088088015 us; error 30 us; skew 500 ppm
Aug 11 07:41:28 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434088, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:28.090771 1838 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:41:28.091593 1838 webserver.cc:460] Webserver started at http://127.18.72.254:37851/ using document root <none> and password file <none>
I20260811 07:41:28.092222 1838 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:28.092293 1838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:28.096081 1838 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260811 07:41:28.098151 1856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:28.099346 1838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:41:28.099448 1838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "b3494d0d27f2400d8c74102a5ee60635"
format_stamp: "Formatted at 2026-08-11 07:41:26 on dist-test-slave-zkpd"
server_key: "fa4a3582046cae680844e1fa94a17ff0"
server_key_iv: "b19572a21e841f2f6058f7e071935940"
server_key_version: "encryptionkey@0"
I20260811 07:41:28.100145 1838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:28.110888 1838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:28.114298 1838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:28.114504 1838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:28.122342 1838 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:37847
I20260811 07:41:28.122643 1913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:37847 every 8 connection(s)
I20260811 07:41:28.123639 1838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:41:28.128171 1914 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:41:28.130123 1914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Bootstrap starting.
I20260811 07:41:28.133561 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 1838
I20260811 07:41:28.134070 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:39709
--local_ip_for_outbound_sockets=127.18.72.193
--tserver_master_addrs=127.18.72.254:37847
--webserver_port=43449
--webserver_interface=127.18.72.193
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:41:28.153615 1914 log.cc:824] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:28.165035 1914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:41:28.165467 1914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Bootstrap complete.
I20260811 07:41:28.170992 1914 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:28.171517 1914 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3494d0d27f2400d8c74102a5ee60635, State: Initialized, Role: FOLLOWER
I20260811 07:41:28.172098 1914 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:28.172246 1914 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:41:28.172333 1914 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:41:28.172438 1914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:41:28.174844 1914 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:28.175208 1914 leader_election.cc:302] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3494d0d27f2400d8c74102a5ee60635; no voters:
I20260811 07:41:28.175544 1914 leader_election.cc:288] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:41:28.175697 1920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:41:28.176002 1920 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 2 LEADER]: Becoming Leader. State: Replica: b3494d0d27f2400d8c74102a5ee60635, State: Running, Role: LEADER
I20260811 07:41:28.176301 1920 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:28.176802 1914 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:28.177285 1922 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3494d0d27f2400d8c74102a5ee60635. Latest consensus state: current_term: 2 leader_uuid: "b3494d0d27f2400d8c74102a5ee60635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } } }
I20260811 07:41:28.177429 1922 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:28.177675 1921 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b3494d0d27f2400d8c74102a5ee60635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } } }
I20260811 07:41:28.177757 1921 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:28.177856 1925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:41:28.182574 1925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:41:28.183211 1925 catalog_manager.cc:1259] Loaded cluster ID: bcab5ec127e047299487ce1c9de1be82
I20260811 07:41:28.183271 1925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:41:28.184741 1925 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 07:41:28.189647 1925 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 07:41:28.194869 1925 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:41:28.195568 1925 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Loaded TSK: 0
I20260811 07:41:28.196765 1925 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:41:28.264846 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:28.265116 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:28.265192 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:28.269174 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:28.269254 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:28.269346 1918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:41:28.274542 1918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:39709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=43449
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:28.275547 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:28.276628 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:28.286756 1944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:28.286764 1943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:28.287161 1918 server_base.cc:1034] running on GCE node
W20260811 07:41:28.287334 1946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:28.287815 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:28.288565 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:28.289793 1918 hybrid_clock.cc:648] HybridClock initialized: now 1786434088289772 us; error 28 us; skew 500 ppm
Aug 11 07:41:28 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434088, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:28.292915 1918 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:41:28.293951 1918 webserver.cc:460] Webserver started at http://127.18.72.193:43449/ using document root <none> and password file <none>
I20260811 07:41:28.294539 1918 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:28.294647 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:28.298758 1918 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:41:28.301121 1953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:28.302474 1918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:41:28.302591 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "79693eda0b5a4c20acf9150cf322f939"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "ca84fa8d58cdbecc670d2540f39ca0be"
server_key_iv: "b3d5908c7f1977e933fe340ce49afb15"
server_key_version: "encryptionkey@0"
I20260811 07:41:28.303233 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:28.319192 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:28.322299 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:28.322494 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:28.323130 1918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:28.324352 1960 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 07:41:28.328424 1918 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:41:28.328485 1918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260811 07:41:28.328558 1918 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:41:28.330215 1918 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:41:28.330258 1918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 07:41:28.330564 1960 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Bootstrap starting.
I20260811 07:41:28.340956 1918 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:39709
I20260811 07:41:28.340992 2078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:39709 every 8 connection(s)
I20260811 07:41:28.341868 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:41:28.344192 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 1918
I20260811 07:41:28.344682 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:42025
--local_ip_for_outbound_sockets=127.18.72.194
--tserver_master_addrs=127.18.72.254:37847
--webserver_port=35645
--webserver_interface=127.18.72.194
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:41:28.350656 1960 log.cc:824] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Log is configured to *not* fsync() on all Append() calls
Aug 11 07:41:28 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434088, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:28.358886 1960 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:41:28.359335 1960 tablet_bootstrap.cc:492] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Bootstrap complete.
I20260811 07:41:28.360757 1960 ts_tablet_manager.cc:1397] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Time spent bootstrapping tablet: real 0.030s user 0.022s sys 0.007s
I20260811 07:41:28.362288 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.343846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:55569)
0811 07:41:28.344777 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.344831 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:28.345630 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:28.347369 (+ 1739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:28.347381 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:28.347924 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:28.348610 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.348634 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.350450 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.350456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.352088 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.352104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.353067 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.353074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.353344 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.354097 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:28.354125 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:28.356207 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:28.358806 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:28.358812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:28.358827 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:28.359162 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:28.359629 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:28.359632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:28.359634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:28.359810 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:28.360609 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:28.360614 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:28.360890 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 07:41:28.361074 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 07:41:28.361387 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":11,"thread_start_us":156,"threads_started":1}
I20260811 07:41:28.362326 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.343992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:55569 (local address 127.18.72.254:37847)
0811 07:41:28.344496 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.344516 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.344949 (+ 433us) server_negotiation.cc:407] Connection header received
0811 07:41:28.346104 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.346117 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.346465 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.346862 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:28.348821 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.350261 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.352233 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.352897 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.353080 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.356445 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:28.356452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:28.356468 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:28.356500 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:28.358669 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.359307 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.359317 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.359322 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.359518 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.359955 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.359960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.359962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.360422 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:28.360539 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.361238 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.361531 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":107,"threads_started":1}
I20260811 07:41:28.363600 1960 raft_consensus.cc:348] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:28.364073 1960 raft_consensus.cc:729] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79693eda0b5a4c20acf9150cf322f939, State: Initialized, Role: FOLLOWER
I20260811 07:41:28.364578 1960 consensus_queue.cc:260] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } } peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } }
I20260811 07:41:28.365535 1960 ts_tablet_manager.cc:1428] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 07:41:28.365535 2079 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37847
I20260811 07:41:28.365809 2079 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:28.366284 2079 heartbeater.cc:511] Master 127.18.72.254:37847 requested a full tablet report, sending...
I20260811 07:41:28.368624 1878 ts_manager.cc:194] Registered new tserver with Master: 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709)
I20260811 07:41:28.371922 1878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:55569
I20260811 07:41:28.374548 2079 heartbeater.cc:503] Master 127.18.72.254:37847 was elected leader, sending a full tablet report...
Aug 11 07:41:28 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434088, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:41:28.382011 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.371366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39709 (local address 127.0.0.1:51006)
0811 07:41:28.372076 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.372117 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:28.372302 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:28.372986 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:28.372992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:28.373296 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:28.373397 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.373403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.374385 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.374389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.375220 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.375230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.376133 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.376136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.376247 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.376823 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:28.376837 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:28.378470 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:28.380273 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:28.380283 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:28.380289 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:28.380522 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:28.380833 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:28.380837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:28.380841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:28.380931 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:28.381377 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:28.381382 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:28.381506 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 07:41:28.381645 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 07:41:28.381835 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":100,"threads_started":1}
I20260811 07:41:28.382050 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.371460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.18.72.193:39709)
0811 07:41:28.371796 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.371807 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.372205 (+ 398us) server_negotiation.cc:407] Connection header received
0811 07:41:28.372482 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.372488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.372682 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.372790 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:28.373518 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.374262 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.375372 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.375880 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.376027 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.378614 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:28.378623 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:28.378632 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:28.378653 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:28.380163 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.380651 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.380656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.380657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.380737 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.381024 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.381028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.381030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.381254 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:28.381397 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.381729 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.381899 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
I20260811 07:41:28.383244 2010 tablet_service.cc:1514] Processing DeleteTablet for tablet 2182831c14fb463cad1520c15d8c6deb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:27 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:51006
I20260811 07:41:28.384351 2089 tablet_replica.cc:330] stopping tablet replica
I20260811 07:41:28.384637 2089 raft_consensus.cc:2229] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:41:28.384881 2089 raft_consensus.cc:2258] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:28.385808 2089 ts_tablet_manager.cc:1905] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:41:28.389312 2089 ts_tablet_manager.cc:1918] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:28.389432 2089 log.cc:1196] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/2182831c14fb463cad1520c15d8c6deb
I20260811 07:41:28.389724 2089 ts_tablet_manager.cc:1939] T 2182831c14fb463cad1520c15d8c6deb P 79693eda0b5a4c20acf9150cf322f939: Deleting consensus metadata
I20260811 07:41:28.391289 1857 catalog_manager.cc:4841] TS 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709): tablet 2182831c14fb463cad1520c15d8c6deb (table test-table [id=c786517bf290425aae599d0d9dc6ea10]) successfully deleted
W20260811 07:41:28.466637 2084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:28.466872 2084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:28.466913 2084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:28.470844 2084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:28.470891 2084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:28.470943 2084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:41:28.475996 2084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:42025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=35645
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:28.476987 2084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:28.478098 2084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:28.487737 2098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:28.487738 2095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:28.487742 2096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:28.488065 2084 server_base.cc:1034] running on GCE node
I20260811 07:41:28.488519 2084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:28.489087 2084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:28.490272 2084 hybrid_clock.cc:648] HybridClock initialized: now 1786434088490233 us; error 50 us; skew 500 ppm
Aug 11 07:41:28 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434088, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:28.492952 2084 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:41:28.493836 2084 webserver.cc:460] Webserver started at http://127.18.72.194:35645/ using document root <none> and password file <none>
I20260811 07:41:28.494451 2084 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:28.494554 2084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:28.498302 2084 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260811 07:41:28.500334 2106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:28.501556 2084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:41:28.501698 2084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "7e479c50b025432bb94681789b3eb54a"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "37000ef11f5ac9b161f9ef6d523c7aff"
server_key_iv: "bb7df9e5930a2f726e2b40e316a26888"
server_key_version: "encryptionkey@0"
I20260811 07:41:28.502255 2084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:28.517683 2084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:28.520381 2084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:28.520574 2084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:28.521147 2084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:28.521950 2084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:28.522020 2084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:28.522094 2084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:28.522136 2084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:28.531117 2084 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:42025
I20260811 07:41:28.531144 2228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:42025 every 8 connection(s)
I20260811 07:41:28.532137 2084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:41:28.532769 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 2084
I20260811 07:41:28.533245 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:40959
--local_ip_for_outbound_sockets=127.18.72.195
--tserver_master_addrs=127.18.72.254:37847
--webserver_port=44809
--webserver_interface=127.18.72.195
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:41:28 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434088, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:28.547683 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.534059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:36961 (local address 127.18.72.254:37847)
0811 07:41:28.534229 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.534235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.535096 (+ 861us) server_negotiation.cc:407] Connection header received
0811 07:41:28.536641 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.536644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.536695 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.536768 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:28.538445 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.539004 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.540937 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.541162 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.541242 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.544147 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:28.544150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:28.544152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:28.544173 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:28.545622 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.546144 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.546148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.546149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.546222 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.546584 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.546587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.546589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.546762 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:28.546868 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.547372 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.547502 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:41:28.548522 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.534410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:36961)
0811 07:41:28.534928 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.534988 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:28.536406 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:28.536970 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:28.536983 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:28.537513 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:28.538240 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.538266 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.539141 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.539149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.540799 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.540815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.541323 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.541330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.541580 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.542192 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:28.542217 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:28.543978 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:28.545742 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:28.545748 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:28.545762 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:28.546025 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:28.546334 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:28.546338 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:28.546340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:28.546479 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:28.546869 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:28.546876 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:28.547128 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 07:41:28.547305 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 07:41:28.547613 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":121,"threads_started":1}
I20260811 07:41:28.549683 2229 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37847
I20260811 07:41:28.549933 2229 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:28.550477 2229 heartbeater.cc:511] Master 127.18.72.254:37847 requested a full tablet report, sending...
I20260811 07:41:28.551522 1878 ts_manager.cc:194] Registered new tserver with Master: 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025)
I20260811 07:41:28.552142 1878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:36961
W20260811 07:41:28.649166 2232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:28.649400 2232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:28.649477 2232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:28.653379 2232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:28.653427 2232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:28.653538 2232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:41:28.658754 2232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:40959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=44809
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:28.659695 2232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:28.660727 2232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:28.669886 2237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:28.669886 2240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:28.669920 2238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:28.670100 2232 server_base.cc:1034] running on GCE node
I20260811 07:41:28.670605 2232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:28.671118 2232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:28.672300 2232 hybrid_clock.cc:648] HybridClock initialized: now 1786434088672284 us; error 41 us; skew 500 ppm
Aug 11 07:41:28 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434088, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:28.674952 2232 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:41:28.675786 2232 webserver.cc:460] Webserver started at http://127.18.72.195:44809/ using document root <none> and password file <none>
I20260811 07:41:28.676353 2232 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:28.676463 2232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:28.680275 2232 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 07:41:28.682266 2248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:28.683318 2232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:41:28.683456 2232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "66711bb2dc1144cfbf72b2d12d181c91"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "16d00f7eb694f651e1f21772059c3538"
server_key_iv: "66d00abddc017dc9664f464c3efd1bac"
server_key_version: "encryptionkey@0"
I20260811 07:41:28.684057 2232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:28.711872 2232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:28.715065 2232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:28.715202 2232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:28.715696 2232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:28.716495 2232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:28.716539 2232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:28.716578 2232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:28.716600 2232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:28.725529 2232 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:40959
I20260811 07:41:28.725566 2371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:40959 every 8 connection(s)
I20260811 07:41:28.726454 2232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:41:28.731174 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 2232
Aug 11 07:41:28 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434088, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:28.740445 2375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.731862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.18.72.254:37847)
0811 07:41:28.732149 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.732152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.732380 (+ 228us) server_negotiation.cc:407] Connection header received
0811 07:41:28.732495 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.732497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.732534 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.732613 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:28.733375 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.734136 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.735186 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.735536 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.735638 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.736985 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:28.736987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:28.736989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:28.737023 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:28.738780 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.739215 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.739218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.739219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.739289 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.739568 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.739572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.739575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.739761 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:28.739973 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.740150 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.740265 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":145,"threads_started":1}
I20260811 07:41:28.740799 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.728139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:48945 (local address 127.18.72.254:37847)
0811 07:41:28.728274 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.728277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.729082 (+ 805us) server_negotiation.cc:407] Connection header received
0811 07:41:28.730447 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.730450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.730509 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.730605 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:28.731883 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.732694 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.734051 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.734322 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.734428 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.736899 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:28.736905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:28.736911 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:28.736941 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:28.738627 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.739172 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.739178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.739181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.739267 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:28.739629 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:28.739632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:28.739633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:28.739786 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:28.739941 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.740540 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.740656 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:41:28.741799 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.728508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:48945)
0811 07:41:28.728945 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.728985 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:28.730237 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:28.730785 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:28.730792 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:28.731108 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:28.731694 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.731716 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.732811 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.732814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.733930 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.733941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.734469 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.734472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.734608 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.735158 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:28.735180 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:28.736733 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:28.738740 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:28.738747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:28.738760 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:28.739076 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:28.739358 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:28.739363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:28.739368 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:28.739509 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:28.739959 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:28.739965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:28.740263 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 07:41:28.740468 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 07:41:28.740778 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":109,"threads_started":1}
I20260811 07:41:28.742827 2372 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37847
I20260811 07:41:28.743064 2372 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:28.743523 2372 heartbeater.cc:511] Master 127.18.72.254:37847 requested a full tablet report, sending...
I20260811 07:41:28.744508 1878 ts_manager.cc:194] Registered new tserver with Master: 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959)
I20260811 07:41:28.745043 1878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:48945
I20260811 07:41:28.753643 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:41:28.757953 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.754257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34430 (local address 127.18.72.254:37847)
0811 07:41:28.754496 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.754500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.754663 (+ 163us) server_negotiation.cc:407] Connection header received
0811 07:41:28.754843 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.754846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.754948 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.755046 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:41:28.755281 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.755731 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.756649 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.756863 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.756968 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.757265 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:41:28.757575 (+ 310us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:41:28.757697 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.757777 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.757832 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260811 07:41:28.758512 1878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:41:28.761773 1878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:41:28.769049 2010 tablet_service.cc:1467] Processing CreateTablet for tablet 3867788e11cf4c75a04c4c3adb7854e0 (DEFAULT_TABLE table=test-table [id=9396cf8c8b0744d4a1a19aedac4fbc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:28.769712 2010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3867788e11cf4c75a04c4c3adb7854e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:28.772584 1960 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Bootstrap starting.
I20260811 07:41:28.772926 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.768085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:40959 (local address 127.0.0.1:60820)
0811 07:41:28.768343 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.768356 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:28.768449 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:28.769363 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:28.769366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:28.769387 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:28.769465 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.769471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.770705 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.770710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.771524 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.771534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.772546 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.772551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.772639 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.772684 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 07:41:28.772748 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 07:41:28.772817 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
I20260811 07:41:28.773479 1960 tablet_bootstrap.cc:654] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:28.773540 2383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.768587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.18.72.195:40959)
0811 07:41:28.768959 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.768970 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.768991 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:41:28.769055 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.769060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.769237 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.769385 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:28.769558 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.770565 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.771656 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.772417 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.772640 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.773160 (+ 520us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.773289 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.773410 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":158,"threads_started":1}
I20260811 07:41:28.773864 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.767714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:42025 (local address 127.0.0.1:43696)
0811 07:41:28.767827 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.767843 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:28.767933 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:28.769000 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:28.769003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:28.769022 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:28.769108 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.769114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.770849 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.770853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.771972 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.771983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.773451 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.773455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.773557 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.773583 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:41:28.773638 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 07:41:28.773742 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 07:41:28.774528 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.768065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43696 (local address 127.18.72.194:42025)
0811 07:41:28.768432 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.768447 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.768470 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:41:28.768551 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.768556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.768774 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.769021 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:28.769200 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.770706 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.772104 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.773323 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.774117 (+ 794us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.774257 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.774309 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.774403 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":146,"threads_started":1}
I20260811 07:41:28.774940 1960 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: No bootstrap required, opened a new log
I20260811 07:41:28.775056 1960 ts_tablet_manager.cc:1397] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 07:41:28.775496 1960 raft_consensus.cc:348] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.775671 1960 raft_consensus.cc:374] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:28.775712 1960 raft_consensus.cc:729] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79693eda0b5a4c20acf9150cf322f939, State: Initialized, Role: FOLLOWER
I20260811 07:41:28.776034 1960 consensus_queue.cc:260] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.776307 1960 ts_tablet_manager.cc:1428] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 07:41:28.776621 2301 tablet_service.cc:1467] Processing CreateTablet for tablet 3867788e11cf4c75a04c4c3adb7854e0 (DEFAULT_TABLE table=test-table [id=9396cf8c8b0744d4a1a19aedac4fbc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:28.777161 2159 tablet_service.cc:1467] Processing CreateTablet for tablet 3867788e11cf4c75a04c4c3adb7854e0 (DEFAULT_TABLE table=test-table [id=9396cf8c8b0744d4a1a19aedac4fbc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:28.777562 2301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3867788e11cf4c75a04c4c3adb7854e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:28.778038 2159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3867788e11cf4c75a04c4c3adb7854e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:28.785084 2386 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: Bootstrap starting.
I20260811 07:41:28.785697 2385 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: Bootstrap starting.
I20260811 07:41:28.787961 2385 tablet_bootstrap.cc:654] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:28.788712 2385 log.cc:824] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:28.790346 2386 tablet_bootstrap.cc:654] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:28.790524 2385 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: No bootstrap required, opened a new log
I20260811 07:41:28.790721 2385 ts_tablet_manager.cc:1397] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:41:28.790943 2386 log.cc:824] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:28.792642 2386 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: No bootstrap required, opened a new log
I20260811 07:41:28.792784 2386 ts_tablet_manager.cc:1397] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 07:41:28.793972 2385 raft_consensus.cc:348] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.794200 2385 raft_consensus.cc:374] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:28.794262 2385 raft_consensus.cc:729] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e479c50b025432bb94681789b3eb54a, State: Initialized, Role: FOLLOWER
I20260811 07:41:28.794736 2385 consensus_queue.cc:260] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.795475 2229 heartbeater.cc:503] Master 127.18.72.254:37847 was elected leader, sending a full tablet report...
I20260811 07:41:28.795754 2385 ts_tablet_manager.cc:1428] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 07:41:28.796044 2386 raft_consensus.cc:348] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.796294 2386 raft_consensus.cc:374] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:28.796365 2386 raft_consensus.cc:729] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66711bb2dc1144cfbf72b2d12d181c91, State: Initialized, Role: FOLLOWER
I20260811 07:41:28.796809 2386 consensus_queue.cc:260] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.797473 2372 heartbeater.cc:503] Master 127.18.72.254:37847 was elected leader, sending a full tablet report...
I20260811 07:41:28.797792 2386 ts_tablet_manager.cc:1428] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 07:41:28.843114 2081 tablet.cc:2369] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:41:28.868147 2390 raft_consensus.cc:482] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:28.868325 2390 raft_consensus.cc:504] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.869385 2390 leader_election.cc:288] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959), 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709)
I20260811 07:41:28.872913 2383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.869705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:57663 (local address 127.18.72.195:40959)
0811 07:41:28.869857 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.869861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.869878 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:41:28.869934 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.869939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.869987 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.870062 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:28.870382 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.870882 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.871884 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.872346 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.872479 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.872536 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.872720 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.872797 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:41:28.872913 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.869599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:40959 (local address 127.18.72.194:57663)
0811 07:41:28.869720 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.869753 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:28.869847 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:28.870099 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:28.870103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:28.870126 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:28.870259 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.870264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.870988 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.870992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.871732 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.871741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.872465 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.872469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.872582 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.872612 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 07:41:28.872702 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 07:41:28.872773 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 07:41:28.873517 2323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3867788e11cf4c75a04c4c3adb7854e0" candidate_uuid: "7e479c50b025432bb94681789b3eb54a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66711bb2dc1144cfbf72b2d12d181c91" is_pre_election: true
I20260811 07:41:28.873646 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.870039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:40809 (local address 127.18.72.193:39709)
0811 07:41:28.870178 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.870185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.870427 (+ 242us) server_negotiation.cc:407] Connection header received
0811 07:41:28.870542 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.870545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.870602 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.870695 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:28.870978 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.871443 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.872603 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.873158 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.873287 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.873439 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.873478 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.873545 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:41:28.873657 2393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.869925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39709 (local address 127.18.72.194:40809)
0811 07:41:28.870325 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.870337 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:28.870458 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:28.870719 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:28.870722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:28.870739 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:28.870869 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.870874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.871560 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.871563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.872477 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:28.872486 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.873288 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.873292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:28.873380 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.873402 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:41:28.873466 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 07:41:28.873528 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":134,"threads_started":1}
I20260811 07:41:28.873881 2323 raft_consensus.cc:2454] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e479c50b025432bb94681789b3eb54a in term 0.
I20260811 07:41:28.874289 2031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3867788e11cf4c75a04c4c3adb7854e0" candidate_uuid: "7e479c50b025432bb94681789b3eb54a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79693eda0b5a4c20acf9150cf322f939" is_pre_election: true
I20260811 07:41:28.874385 2107 leader_election.cc:302] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66711bb2dc1144cfbf72b2d12d181c91, 7e479c50b025432bb94681789b3eb54a; no voters:
I20260811 07:41:28.874575 2031 raft_consensus.cc:2454] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e479c50b025432bb94681789b3eb54a in term 0.
I20260811 07:41:28.874667 2390 raft_consensus.cc:2781] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:41:28.874771 2390 raft_consensus.cc:482] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:41:28.874845 2390 raft_consensus.cc:3037] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:28.875941 2390 raft_consensus.cc:504] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.876281 2390 leader_election.cc:288] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [CANDIDATE]: Term 1 election: Requested vote from peers 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959), 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709)
I20260811 07:41:28.876690 2323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3867788e11cf4c75a04c4c3adb7854e0" candidate_uuid: "7e479c50b025432bb94681789b3eb54a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66711bb2dc1144cfbf72b2d12d181c91"
I20260811 07:41:28.876716 2031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3867788e11cf4c75a04c4c3adb7854e0" candidate_uuid: "7e479c50b025432bb94681789b3eb54a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79693eda0b5a4c20acf9150cf322f939"
I20260811 07:41:28.876842 2323 raft_consensus.cc:3037] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:28.876844 2031 raft_consensus.cc:3037] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:28.878063 2031 raft_consensus.cc:2454] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e479c50b025432bb94681789b3eb54a in term 1.
I20260811 07:41:28.878084 2323 raft_consensus.cc:2454] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e479c50b025432bb94681789b3eb54a in term 1.
I20260811 07:41:28.878346 2107 leader_election.cc:302] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79693eda0b5a4c20acf9150cf322f939, 7e479c50b025432bb94681789b3eb54a; no voters:
I20260811 07:41:28.878523 2390 raft_consensus.cc:2781] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:28.878741 2390 raft_consensus.cc:686] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 1 LEADER]: Becoming Leader. State: Replica: 7e479c50b025432bb94681789b3eb54a, State: Running, Role: LEADER
I20260811 07:41:28.879066 2390 consensus_queue.cc:237] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:28.882119 1877 catalog_manager.cc:5495] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e479c50b025432bb94681789b3eb54a (127.18.72.194). New cstate: current_term: 1 leader_uuid: "7e479c50b025432bb94681789b3eb54a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } health_report { overall_health: UNKNOWN } } }
I20260811 07:41:28.898272 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.894723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.18.72.194:42025)
0811 07:41:28.894845 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:28.894850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:28.894942 (+ 92us) server_negotiation.cc:407] Connection header received
0811 07:41:28.895132 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:28.895134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:28.895187 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:28.895294 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:41:28.895524 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.896069 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.896893 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:28.897109 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:28.897219 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:28.897514 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:41:28.897896 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:41:28.898025 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:28.898089 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:41:28.898143 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:41:28.903908 2031 raft_consensus.cc:1261] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Refusing update from remote peer 7e479c50b025432bb94681789b3eb54a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:28.903944 2323 raft_consensus.cc:1261] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 FOLLOWER]: Refusing update from remote peer 7e479c50b025432bb94681789b3eb54a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:28.904587 2394 consensus_queue.cc:1035] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:28.904724 2390 consensus_queue.cc:1035] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:28.915059 2398 mvcc.cc:205] Tried to move back new op lower bound from 7317234028143345664 to 7317234028051095552. Current Snapshot: MvccSnapshot[applied={T|T < 7317234028143345664 or (T in {7317234028143345664})}]
I20260811 07:41:28.919438 1878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:41:28.919567 1878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:41:28.921829 1878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Sending DeleteTablet for 3 replicas of tablet 3867788e11cf4c75a04c4c3adb7854e0
I20260811 07:41:28.922528 2159 tablet_service.cc:1514] Processing DeleteTablet for tablet 3867788e11cf4c75a04c4c3adb7854e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:28 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43696
I20260811 07:41:28.922700 2010 tablet_service.cc:1514] Processing DeleteTablet for tablet 3867788e11cf4c75a04c4c3adb7854e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:28 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:51006
I20260811 07:41:28.922737 2301 tablet_service.cc:1514] Processing DeleteTablet for tablet 3867788e11cf4c75a04c4c3adb7854e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:28 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:60820
I20260811 07:41:28.923144 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 1918
I20260811 07:41:28.928119 2408 tablet_replica.cc:330] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: stopping tablet replica
I20260811 07:41:28.928485 2408 raft_consensus.cc:2229] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:41:28.928923 2408 raft_consensus.cc:2258] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:28.923209 2410 tablet_replica.cc:330] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: stopping tablet replica
I20260811 07:41:28.930291 2410 raft_consensus.cc:2229] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:41:28.930624 2410 raft_consensus.cc:2258] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:28.930863 2408 ts_tablet_manager.cc:1905] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:41:28.931980 2410 ts_tablet_manager.cc:1905] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:41:28.934458 2408 ts_tablet_manager.cc:1918] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:28.934568 2408 log.cc:1196] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/3867788e11cf4c75a04c4c3adb7854e0
I20260811 07:41:28.934808 2410 ts_tablet_manager.cc:1918] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:28.934924 2408 ts_tablet_manager.cc:1939] T 3867788e11cf4c75a04c4c3adb7854e0 P 7e479c50b025432bb94681789b3eb54a: Deleting consensus metadata
I20260811 07:41:28.935030 2410 log.cc:1196] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/3867788e11cf4c75a04c4c3adb7854e0
I20260811 07:41:28.935433 2410 ts_tablet_manager.cc:1939] T 3867788e11cf4c75a04c4c3adb7854e0 P 66711bb2dc1144cfbf72b2d12d181c91: Deleting consensus metadata
I20260811 07:41:28.936208 1860 catalog_manager.cc:4841] TS 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025): tablet 3867788e11cf4c75a04c4c3adb7854e0 (table test-table [id=9396cf8c8b0744d4a1a19aedac4fbc39]) successfully deleted
I20260811 07:41:28.936676 1857 catalog_manager.cc:4841] TS 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959): tablet 3867788e11cf4c75a04c4c3adb7854e0 (table test-table [id=9396cf8c8b0744d4a1a19aedac4fbc39]) successfully deleted
W20260811 07:41:28.938666 1857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:39709 (error 108)
I20260811 07:41:28.939277 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 2084
I20260811 07:41:28.941190 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:28.939440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39709 (local address 127.0.0.1:51012)
0811 07:41:28.940996 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:28.941084 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1445}
W20260811 07:41:28.942196 1857 catalog_manager.cc:4584] TS 79693eda0b5a4c20acf9150cf322f939 (127.18.72.193:39709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3867788e11cf4c75a04c4c3adb7854e0: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39709: connect: Connection refused (error 111)
I20260811 07:41:28.948256 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 2232
I20260811 07:41:28.955945 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 1838
I20260811 07:41:28.963989 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:37847
--webserver_interface=127.18.72.254
--webserver_port=37851
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:37847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:29.078346 2412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:29.078655 2412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:29.078758 2412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:29.082857 2412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:29.082907 2412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:29.082926 2412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:29.082942 2412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:29.082957 2412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:29.087775 2412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:37847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:37847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=37851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:29.088692 2412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:29.089614 2412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:29.097714 2420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:29.097700 2417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:29.097690 2418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:29.098028 2412 server_base.cc:1034] running on GCE node
I20260811 07:41:29.098603 2412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:29.099619 2412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:29.100811 2412 hybrid_clock.cc:648] HybridClock initialized: now 1786434089100780 us; error 50 us; skew 500 ppm
Aug 11 07:41:29 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434089, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:29.103497 2412 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:41:29.104422 2412 webserver.cc:460] Webserver started at http://127.18.72.254:37851/ using document root <none> and password file <none>
I20260811 07:41:29.104949 2412 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:29.105038 2412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:29.108675 2412 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:41:29.110664 2428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:29.111939 2412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:41:29.112074 2412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "b3494d0d27f2400d8c74102a5ee60635"
format_stamp: "Formatted at 2026-08-11 07:41:26 on dist-test-slave-zkpd"
server_key: "fa4a3582046cae680844e1fa94a17ff0"
server_key_iv: "b19572a21e841f2f6058f7e071935940"
server_key_version: "encryptionkey@0"
I20260811 07:41:29.112807 2412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:29.125941 2412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:29.129105 2412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:29.129263 2412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:29.136471 2412 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:37847
I20260811 07:41:29.136479 2484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:37847 every 8 connection(s)
I20260811 07:41:29.137524 2412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:41:29.140166 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 2412
I20260811 07:41:29.140614 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:39709
--local_ip_for_outbound_sockets=127.18.72.193
--tserver_master_addrs=127.18.72.254:37847
--webserver_port=43449
--webserver_interface=127.18.72.193
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:41:29.141974 2485 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:41:29.143577 2485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Bootstrap starting.
I20260811 07:41:29.169819 2485 log.cc:824] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:29.184297 2485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:41:29.184701 2485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635: Bootstrap complete.
I20260811 07:41:29.190827 2485 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:29.191342 2485 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3494d0d27f2400d8c74102a5ee60635, State: Initialized, Role: FOLLOWER
I20260811 07:41:29.191882 2485 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:29.192090 2485 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:41:29.192168 2485 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:41:29.192265 2485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 2 FOLLOWER]: Advancing to term 3
I20260811 07:41:29.194744 2485 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:29.195098 2485 leader_election.cc:302] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3494d0d27f2400d8c74102a5ee60635; no voters:
I20260811 07:41:29.195430 2485 leader_election.cc:288] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 07:41:29.195566 2489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 07:41:29.195813 2489 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [term 3 LEADER]: Becoming Leader. State: Replica: b3494d0d27f2400d8c74102a5ee60635, State: Running, Role: LEADER
I20260811 07:41:29.196144 2489 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } }
I20260811 07:41:29.196682 2485 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:29.197499 2490 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b3494d0d27f2400d8c74102a5ee60635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } } }
I20260811 07:41:29.197624 2490 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:29.197965 2491 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3494d0d27f2400d8c74102a5ee60635. Latest consensus state: current_term: 3 leader_uuid: "b3494d0d27f2400d8c74102a5ee60635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3494d0d27f2400d8c74102a5ee60635" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37847 } } }
I20260811 07:41:29.198064 2491 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3494d0d27f2400d8c74102a5ee60635 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:29.198601 2499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:41:29.203593 2499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:41:29.204249 2499 catalog_manager.cc:1259] Loaded cluster ID: bcab5ec127e047299487ce1c9de1be82
I20260811 07:41:29.204310 2499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 07:41:29.205267 2499 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f89993669ad google::LogMessage::Fail() at ??:0
@ 0x7f899936988a google::LogMessage::SendToLog() at ??:0
@ 0x7f89993664de google::LogMessage::Flush() at ??:0
@ 0x7f8999367a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f89a3ca5fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f89a3ca6d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f89a3ca3579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f89a3ce7640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f899b5b83d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f899b5b8c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f899b5ba3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f899b5a9512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f899cbed6db start_thread at ??:0
@ 0x7f8997e6461f clone at ??:0
W20260811 07:41:29.288570 2487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:29.288841 2487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:29.288930 2487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:29.292954 2487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:29.293037 2487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:29.293126 2487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:41:29.298480 2487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:39709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=43449
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:29.299554 2487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:29.300657 2487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:29.310585 2529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:29.310581 2528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:29.310581 2531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:29.310853 2487 server_base.cc:1034] running on GCE node
I20260811 07:41:29.311371 2487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:29.312152 2487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:29.313347 2487 hybrid_clock.cc:648] HybridClock initialized: now 1786434089313308 us; error 52 us; skew 500 ppm
Aug 11 07:41:29 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434089, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:29.316512 2487 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:41:29.317390 2487 webserver.cc:460] Webserver started at http://127.18.72.193:43449/ using document root <none> and password file <none>
I20260811 07:41:29.317975 2487 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:29.318037 2487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:29.322063 2487 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:41:29.324755 2540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:29.326098 2487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:41:29.326246 2487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "79693eda0b5a4c20acf9150cf322f939"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "ca84fa8d58cdbecc670d2540f39ca0be"
server_key_iv: "b3d5908c7f1977e933fe340ce49afb15"
server_key_version: "encryptionkey@0"
I20260811 07:41:29.326854 2487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:29.342303 2487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:29.345430 2487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:29.345737 2487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:29.346388 2487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:29.347679 2548 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 07:41:29.353581 2487 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:41:29.353673 2487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 07:41:29.353773 2487 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:41:29.355420 2487 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:41:29.355516 2487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 07:41:29.355801 2548 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Bootstrap starting.
I20260811 07:41:29.366355 2487 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:39709
I20260811 07:41:29.366510 2663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:39709 every 8 connection(s)
I20260811 07:41:29.367332 2487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:41:29.369375 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 2487
I20260811 07:41:29.369980 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:42025
--local_ip_for_outbound_sockets=127.18.72.194
--tserver_master_addrs=127.18.72.254:37847
--webserver_port=35645
--webserver_interface=127.18.72.194
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:41:29.378130 2548 log.cc:824] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:29.388258 2548 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:41:29.388665 2548 tablet_bootstrap.cc:492] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Bootstrap complete.
I20260811 07:41:29.390255 2548 ts_tablet_manager.cc:1397] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Time spent bootstrapping tablet: real 0.035s user 0.027s sys 0.004s
I20260811 07:41:29.393432 2548 raft_consensus.cc:348] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:29.394022 2548 raft_consensus.cc:729] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79693eda0b5a4c20acf9150cf322f939, State: Initialized, Role: FOLLOWER
I20260811 07:41:29.394579 2548 consensus_queue.cc:260] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:29.395576 2548 ts_tablet_manager.cc:1428] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260811 07:41:29.428943 2666 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:41:29.369684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:57671)
0811 07:41:29.372445 (+ 2761us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.372516 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:29.373575 (+ 1059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:29.428162 (+ 54587us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2518,"thread_start_us":150,"threads_started":1}
W20260811 07:41:29.429574 2664 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:37847 (0 consecutive failures): Network error: Failed to ping master at 127.18.72.254:37847: Client connection negotiation failed: client connection to 127.18.72.254:37847: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 07:41:29.430308 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:29.430048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:45835)
0811 07:41:29.430174 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.430248 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 07:41:29.431006 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:29.430800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:41755)
0811 07:41:29.430903 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.430948 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 07:41:29.431238 2664 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:41:29.496639 2667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:29.496883 2667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:29.496940 2667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:29.500876 2667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:29.500921 2667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:29.500973 2667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:41:29.505924 2667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:42025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=35645
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:29.506994 2667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:29.507970 2667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:29.517186 2676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:29.517241 2675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:29.517192 2679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:29.517408 2667 server_base.cc:1034] running on GCE node
I20260811 07:41:29.517918 2667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:29.518565 2667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:29.519743 2667 hybrid_clock.cc:648] HybridClock initialized: now 1786434089519727 us; error 34 us; skew 500 ppm
Aug 11 07:41:29 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434089, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:29.522387 2667 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:41:29.523134 2667 webserver.cc:460] Webserver started at http://127.18.72.194:35645/ using document root <none> and password file <none>
I20260811 07:41:29.523630 2667 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:29.523690 2667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:29.527256 2667 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:41:29.529201 2686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:29.530390 2667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:41:29.530506 2667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "7e479c50b025432bb94681789b3eb54a"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "37000ef11f5ac9b161f9ef6d523c7aff"
server_key_iv: "bb7df9e5930a2f726e2b40e316a26888"
server_key_version: "encryptionkey@0"
I20260811 07:41:29.531069 2667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:29.545953 2667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:29.548887 2667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:29.549067 2667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:29.549618 2667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:29.550402 2667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:29.550449 2667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:29.550487 2667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:29.550503 2667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:29.559440 2667 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:42025
I20260811 07:41:29.559517 2805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:42025 every 8 connection(s)
I20260811 07:41:29.560474 2667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:41:29.560844 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 2667
I20260811 07:41:29.561314 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:40959
--local_ip_for_outbound_sockets=127.18.72.195
--tserver_master_addrs=127.18.72.254:37847
--webserver_port=44809
--webserver_interface=127.18.72.195
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:41:29.564814 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:29.562848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:55969)
0811 07:41:29.563344 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.563467 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":153,"threads_started":1}
W20260811 07:41:29.565263 2806 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:37847 (0 consecutive failures): Network error: Failed to ping master at 127.18.72.254:37847: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
I20260811 07:41:29.565850 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:29.565644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:41217)
0811 07:41:29.565752 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.565799 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 07:41:29.566541 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:29.566338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:45155)
0811 07:41:29.566439 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.566484 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 07:41:29.566740 2806 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:41:29.672053 2808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:29.672278 2808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:29.672318 2808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:29.676103 2808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:29.676147 2808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:29.676215 2808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:41:29.681205 2808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:40959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=44809
--tserver_master_addrs=127.18.72.254:37847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:29.682044 2808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:29.682947 2808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:29.692538 2815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:29.692562 2816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:29.692538 2819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:29.692903 2808 server_base.cc:1034] running on GCE node
I20260811 07:41:29.693320 2808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:29.693926 2808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:29.695111 2808 hybrid_clock.cc:648] HybridClock initialized: now 1786434089695080 us; error 42 us; skew 500 ppm
Aug 11 07:41:29 dist-test-slave-zkpd krb5kdc[1246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434089, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:29.697782 2808 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:41:29.698565 2808 webserver.cc:460] Webserver started at http://127.18.72.195:44809/ using document root <none> and password file <none>
I20260811 07:41:29.699113 2808 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:29.699203 2808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:29.702863 2808 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:41:29.704854 2826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:29.706032 2808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:41:29.706162 2808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "66711bb2dc1144cfbf72b2d12d181c91"
format_stamp: "Formatted at 2026-08-11 07:41:27 on dist-test-slave-zkpd"
server_key: "16d00f7eb694f651e1f21772059c3538"
server_key_iv: "66d00abddc017dc9664f464c3efd1bac"
server_key_version: "encryptionkey@0"
I20260811 07:41:29.706669 2808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:29.747169 2808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:29.750089 2808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:29.750288 2808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:29.750878 2808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:29.751698 2808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:29.751767 2808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:29.751842 2808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:29.751883 2808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:29.761550 2808 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:40959
I20260811 07:41:29.761580 2945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:40959 every 8 connection(s)
I20260811 07:41:29.762518 2808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:41:29.766134 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:29.764489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:34311)
0811 07:41:29.764934 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.765045 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":129,"threads_started":1}
W20260811 07:41:29.766625 2946 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:37847 (0 consecutive failures): Network error: Failed to ping master at 127.18.72.254:37847: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
I20260811 07:41:29.767119 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:29.766922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:37941)
0811 07:41:29.767024 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.767068 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 07:41:29.767716 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:29.767509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:52151)
0811 07:41:29.767616 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:29.767660 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 07:41:29.767912 2946 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:41:29.768997 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 2808
I20260811 07:41:30.432263 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:30.431791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:56521)
0811 07:41:30.432096 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:30.432153 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":136,"threads_started":1}
I20260811 07:41:30.567668 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:30.567304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:37387)
0811 07:41:30.567525 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:30.567575 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
I20260811 07:41:30.768751 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:30.768388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:43431)
0811 07:41:30.768608 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:30.768658 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":67,"threads_started":1}
I20260811 07:41:31.113183 2954 raft_consensus.cc:482] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:31.113384 2954 raft_consensus.cc:504] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:31.114535 2954 leader_election.cc:288] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025), 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959)
Aug 11 07:41:31 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434089, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.195@KRBTEST.COM
Aug 11 07:41:31 dist-test-slave-zkpd krb5kdc[1246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434089, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.194@KRBTEST.COM
I20260811 07:41:31.128371 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:31.114782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:40959 (local address 127.18.72.193:35837)
0811 07:41:31.115363 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:31.115379 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:31.115512 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:31.117174 (+ 1662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:31.117180 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:31.117726 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:31.118206 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:31.118222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:31.119531 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:31.119533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:31.120541 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:31.120552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:31.121347 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:31.121350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:31.121434 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:31.122097 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:31.122113 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:31.123584 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:31.125851 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:31.125856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:31.125869 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:31.126132 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:31.126488 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:31.126492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:31.126494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:31.126627 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:31.127265 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:31.127272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:31.127537 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 07:41:31.127742 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 07:41:31.128213 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":293,"spinlock_wait_cycles":25344,"thread_start_us":64,"threads_started":1}
I20260811 07:41:31.128407 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:31.114878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:44189 (local address 127.18.72.194:42025)
0811 07:41:31.115427 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:31.115438 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:31.115475 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:41:31.116268 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:31.116275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:31.116699 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:31.117041 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:31.118380 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:31.119355 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:31.120670 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:31.121126 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:31.121321 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:31.124191 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:31.124215 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:31.124235 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:31.124273 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:31.126105 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:31.126543 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:31.126548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:31.126553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:31.126668 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:31.126948 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:31.126952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:31.126954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:31.127382 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:31.127744 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:31.127939 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 07:41:31.128222 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":119,"threads_started":1}
I20260811 07:41:31.128418 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:31.115045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:35837 (local address 127.18.72.195:40959)
0811 07:41:31.115362 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:31.115392 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:31.115493 (+ 101us) server_negotiation.cc:407] Connection header received
0811 07:41:31.116329 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:31.116342 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:31.116772 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:31.117095 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:31.118386 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:31.119373 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:31.120661 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:31.121224 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:31.121403 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:31.123843 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:31.123848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:31.123859 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:31.123886 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:31.125713 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:31.126252 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:31.126258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:31.126263 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:31.126389 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:31.126733 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:31.126736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:31.126738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:31.127138 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:31.127275 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:31.128018 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 07:41:31.128240 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":110,"threads_started":1}
I20260811 07:41:31.128369 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:31.114694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:42025 (local address 127.18.72.193:44189)
0811 07:41:31.115148 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:31.115163 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:31.115269 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:31.117173 (+ 1904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:31.117180 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:31.117727 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:31.118207 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:31.118221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:31.119505 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:31.119509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:31.120539 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:31.120549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:31.121250 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:31.121254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:31.121418 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:31.122154 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:31.122166 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:31.124021 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:31.126226 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:31.126231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:31.126233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:31.126434 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:31.126769 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:31.126772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:31.126773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:31.126860 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:31.127500 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:31.127506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:31.127611 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 07:41:31.127744 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 07:41:31.128158 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":150,"spinlock_wait_cycles":896,"thread_start_us":71,"threads_started":1}
W20260811 07:41:31.131248 2542 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
W20260811 07:41:31.131836 2544 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
I20260811 07:41:31.131944 2544 leader_election.cc:302] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79693eda0b5a4c20acf9150cf322f939; no voters: 66711bb2dc1144cfbf72b2d12d181c91, 7e479c50b025432bb94681789b3eb54a
I20260811 07:41:31.132172 2954 raft_consensus.cc:2726] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:41:31.433221 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:31.432955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:45521)
0811 07:41:31.433079 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:31.433135 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 07:41:31.568663 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:31.568277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:36031)
0811 07:41:31.568511 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:31.568560 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":82,"threads_started":1}
I20260811 07:41:31.769934 2960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:31.769516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:49379)
0811 07:41:31.769775 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:31.769849 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260811 07:41:32.434657 2961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:32.434109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:41867)
0811 07:41:32.434487 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:32.434555 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":82,"threads_started":1}
I20260811 07:41:32.569927 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:32.569501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:40237)
0811 07:41:32.569762 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:32.569825 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":76,"threads_started":1}
I20260811 07:41:32.771047 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:32.770666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:45649)
0811 07:41:32.770902 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:32.770954 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260811 07:41:32.916898 2964 raft_consensus.cc:482] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:32.917025 2964 raft_consensus.cc:504] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:32.917521 2964 leader_election.cc:288] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025), 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959)
W20260811 07:41:32.918445 2544 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
W20260811 07:41:32.918601 2542 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
I20260811 07:41:32.918699 2542 leader_election.cc:302] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79693eda0b5a4c20acf9150cf322f939; no voters: 66711bb2dc1144cfbf72b2d12d181c91, 7e479c50b025432bb94681789b3eb54a
I20260811 07:41:32.918859 2964 raft_consensus.cc:2726] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:41:33.435822 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:33.435433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:38821)
0811 07:41:33.435671 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:33.435724 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260811 07:41:33.570998 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:33.570610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:60037)
0811 07:41:33.570851 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:33.570918 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260811 07:41:33.772159 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:33.771731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:60445)
0811 07:41:33.772017 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:33.772071 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":79,"threads_started":1}
I20260811 07:41:34.437156 2968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:34.436714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:44781)
0811 07:41:34.437005 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:34.437060 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":90,"threads_started":1}
I20260811 07:41:34.572219 2969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:34.571804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:43537)
0811 07:41:34.572086 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:34.572137 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":145,"threads_started":1}
I20260811 07:41:34.773312 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:34.772915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:34851)
0811 07:41:34.773167 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:34.773224 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260811 07:41:35.295356 2971 raft_consensus.cc:482] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:35.295493 2971 raft_consensus.cc:504] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:35.295993 2971 leader_election.cc:288] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025), 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959)
W20260811 07:41:35.296787 2542 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
W20260811 07:41:35.296885 2544 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
I20260811 07:41:35.296944 2544 leader_election.cc:302] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79693eda0b5a4c20acf9150cf322f939; no voters: 66711bb2dc1144cfbf72b2d12d181c91, 7e479c50b025432bb94681789b3eb54a
I20260811 07:41:35.297094 2971 raft_consensus.cc:2726] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:41:35.438328 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:35.437895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:52161)
0811 07:41:35.438171 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:35.438227 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260811 07:41:35.573266 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:35.572882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:53455)
0811 07:41:35.573125 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:35.573175 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
I20260811 07:41:35.774408 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:35.774025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:55469)
0811 07:41:35.774262 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:35.774320 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260811 07:41:36.439489 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:36.439096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:44015)
0811 07:41:36.439338 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:36.439392 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260811 07:41:36.574415 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:36.573970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:58141)
0811 07:41:36.574271 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:36.574327 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":77,"threads_started":1}
I20260811 07:41:36.775743 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:36.775198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:44039)
0811 07:41:36.775553 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:36.775624 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":106,"threads_started":1}
I20260811 07:41:37.044036 2978 raft_consensus.cc:482] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:37.044173 2978 raft_consensus.cc:504] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:37.044595 2978 leader_election.cc:288] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025), 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959)
W20260811 07:41:37.045382 2542 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
W20260811 07:41:37.045497 2544 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
I20260811 07:41:37.045610 2544 leader_election.cc:302] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79693eda0b5a4c20acf9150cf322f939; no voters: 66711bb2dc1144cfbf72b2d12d181c91, 7e479c50b025432bb94681789b3eb54a
I20260811 07:41:37.045784 2978 raft_consensus.cc:2726] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:41:37.440714 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:37.440289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:50601)
0811 07:41:37.440561 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:37.440616 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":104,"threads_started":1}
I20260811 07:41:37.575668 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:37.575238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:36251)
0811 07:41:37.575514 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:37.575572 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260811 07:41:37.776852 2981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:37.776455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:34905)
0811 07:41:37.776701 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:37.776758 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260811 07:41:38.441890 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:38.441467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:34837)
0811 07:41:38.441730 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:38.441788 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260811 07:41:38.576956 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:38.576482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:42861)
0811 07:41:38.576789 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:38.576853 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":91,"threads_started":1}
I20260811 07:41:38.778262 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:38.777822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:39339)
0811 07:41:38.778099 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:38.778162 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260811 07:41:39.443145 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:39.442740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:43403)
0811 07:41:39.442999 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:39.443051 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
I20260811 07:41:39.578233 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:39.577847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:39393)
0811 07:41:39.578086 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:39.578141 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260811 07:41:39.779629 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:39.779127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:32807)
0811 07:41:39.779398 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:39.779459 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":77,"threads_started":1}
I20260811 07:41:40.444341 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:40.443871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:51405)
0811 07:41:40.444184 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:40.444240 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":80,"threads_started":1}
I20260811 07:41:40.579396 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:40.578996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:49105)
0811 07:41:40.579249 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:40.579304 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260811 07:41:40.781162 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:40.780728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:60681)
0811 07:41:40.781009 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:40.781067 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":82,"threads_started":1}
I20260811 07:41:41.445576 2992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:41.445100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:60991)
0811 07:41:41.445406 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:41.445468 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
I20260811 07:41:41.580583 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:41.580189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:48957)
0811 07:41:41.580437 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:41.580491 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
I20260811 07:41:41.782454 2994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:41.782052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:33217)
0811 07:41:41.782301 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:41.782364 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260811 07:41:41.901662 2995 raft_consensus.cc:482] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:41.901832 2995 raft_consensus.cc:504] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e479c50b025432bb94681789b3eb54a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 42025 } } peers { permanent_uuid: "66711bb2dc1144cfbf72b2d12d181c91" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 40959 } } peers { permanent_uuid: "79693eda0b5a4c20acf9150cf322f939" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39709 } }
I20260811 07:41:41.902323 2995 leader_election.cc:288] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025), 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959)
W20260811 07:41:41.903108 2544 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e479c50b025432bb94681789b3eb54a (127.18.72.194:42025): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
W20260811 07:41:41.903213 2542 leader_election.cc:341] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 66711bb2dc1144cfbf72b2d12d181c91 (127.18.72.195:40959): Not found: Tablet not found: 3867788e11cf4c75a04c4c3adb7854e0
I20260811 07:41:41.903280 2542 leader_election.cc:302] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79693eda0b5a4c20acf9150cf322f939; no voters: 66711bb2dc1144cfbf72b2d12d181c91, 7e479c50b025432bb94681789b3eb54a
I20260811 07:41:41.903421 2995 raft_consensus.cc:2726] T 3867788e11cf4c75a04c4c3adb7854e0 P 79693eda0b5a4c20acf9150cf322f939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:41:42.446825 2996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:42.446378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:36591)
0811 07:41:42.446663 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:42.446718 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
I20260811 07:41:42.581727 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:42.581299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:46373)
0811 07:41:42.581583 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:42.581635 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":101,"threads_started":1}
I20260811 07:41:42.783640 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:42.783202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:49849)
0811 07:41:42.783489 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:42.783547 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260811 07:41:43.448295 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:43.447634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:40569)
0811 07:41:43.448130 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:43.448190 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":133,"threads_started":1}
I20260811 07:41:43.582886 3000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:43.582474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:49649)
0811 07:41:43.582733 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:43.582789 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
I20260811 07:41:43.784740 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:43.784355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:32841)
0811 07:41:43.784594 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:43.784649 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260811 07:41:44.449599 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:44.449198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.193:36591)
0811 07:41:44.449450 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:44.449502 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260811 07:41:44.584153 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:44.583704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.194:49097)
0811 07:41:44.584000 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:44.584055 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":89,"threads_started":1}
I20260811 07:41:44.775170 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 2487
I20260811 07:41:44.784471 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 2667
I20260811 07:41:44.787071 3004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:44.786305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37847 (local address 127.18.72.195:40579)
0811 07:41:44.786852 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:44.786922 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":214,"threads_started":1}
I20260811 07:41:44.795269 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 2808
2026-08-11T07:41:44Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19116 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:41:44 dist-test-slave-zkpd krb5kdc[3007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:41:44 dist-test-slave-zkpd krb5kdc[3007](info): set up 2 sockets
Aug 11 07:41:44 dist-test-slave-zkpd krb5kdc[3007](info): commencing operation
krb5kdc: starting...
W20260811 07:41:45.884825 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:41:45 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:41:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:41:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:41:46.006444 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:42967
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:46.117591 3024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:46.117834 3024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:46.117872 3024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:46.121850 3024 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 07:41:46.121896 3024 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 07:41:46.121912 3024 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 07:41:46.121925 3024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:41:46.121942 3024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:46.121960 3024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:46.121978 3024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:41:46.121999 3024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:41:46.127010 3024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:42967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:46.127985 3024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:46.128930 3024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:46.137162 3030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:46.137216 3024 server_base.cc:1034] running on GCE node
W20260811 07:41:46.137140 3031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:46.137372 3033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:46.137984 3024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:46.138970 3024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:46.140126 3024 hybrid_clock.cc:648] HybridClock initialized: now 1786434106140095 us; error 42 us; skew 500 ppm
Aug 11 07:41:46 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434106, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:46.142786 3024 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:41:46.143647 3024 webserver.cc:460] Webserver started at http://127.18.72.254:41963/ using document root <none> and password file <none>
I20260811 07:41:46.144248 3024 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:46.144357 3024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:46.144582 3024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:46.146569 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "c14340c2c2f14e1c8ca60163e8233673"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "f2d12dfd9575765feada411b44030ade"
server_key_iv: "aaf98d7971dafc863c0e95df1106b5bc"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.147126 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "c14340c2c2f14e1c8ca60163e8233673"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "f2d12dfd9575765feada411b44030ade"
server_key_iv: "aaf98d7971dafc863c0e95df1106b5bc"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.150851 3024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:41:46.153457 3042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.154615 3024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:41:46.154748 3024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "c14340c2c2f14e1c8ca60163e8233673"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "f2d12dfd9575765feada411b44030ade"
server_key_iv: "aaf98d7971dafc863c0e95df1106b5bc"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.154862 3024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:46.171598 3024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:46.174306 3024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:46.174512 3024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:46.181393 3024 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:42967
I20260811 07:41:46.181679 3094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:42967 every 8 connection(s)
I20260811 07:41:46.182374 3024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:41:46.185097 3095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:46.190442 3095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Bootstrap starting.
I20260811 07:41:46.192402 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 3024
I20260811 07:41:46.192500 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:41:46.192809 18723 external_mini_cluster.cc:1428] Setting key d8fb07d7bf5f5c75c0f06b316e2920f4
I20260811 07:41:46.192916 3095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:46.193852 3095 log.cc:824] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:46.196043 3095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: No bootstrap required, opened a new log
I20260811 07:41:46.200348 3095 raft_consensus.cc:348] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:41:46.200580 3095 raft_consensus.cc:374] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:46.200662 3095 raft_consensus.cc:729] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14340c2c2f14e1c8ca60163e8233673, State: Initialized, Role: FOLLOWER
I20260811 07:41:46.201151 3095 consensus_queue.cc:260] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:41:46.201309 3095 raft_consensus.cc:388] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:41:46.201400 3095 raft_consensus.cc:482] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:41:46.201502 3095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 07:41:46 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:46.202461 3095 raft_consensus.cc:504] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:41:46.202807 3095 leader_election.cc:302] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c14340c2c2f14e1c8ca60163e8233673; no voters:
I20260811 07:41:46.203176 3095 leader_election.cc:288] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:41:46.203281 3100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:46.203542 3100 raft_consensus.cc:686] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 1 LEADER]: Becoming Leader. State: Replica: c14340c2c2f14e1c8ca60163e8233673, State: Running, Role: LEADER
I20260811 07:41:46.203950 3100 consensus_queue.cc:237] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:41:46.204595 3095 sys_catalog.cc:564] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:41:46.205256 3100 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c14340c2c2f14e1c8ca60163e8233673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } } }
I20260811 07:41:46.205494 3100 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:46.205945 3110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:41:46.206060 3101 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c14340c2c2f14e1c8ca60163e8233673. Latest consensus state: current_term: 1 leader_uuid: "c14340c2c2f14e1c8ca60163e8233673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } } }
I20260811 07:41:46.206213 3101 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: This master's current role is: LEADER
I20260811 07:41:46.208380 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.193989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44030 (local address 127.18.72.254:42967)
0811 07:41:46.194429 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:46.194444 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:46.194477 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:41:46.195281 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:46.195306 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:46.195690 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:46.196071 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:46.196804 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.198319 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.199157 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.199959 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.200117 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.202716 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:46.202721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:46.202736 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:46.202762 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:46.205025 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.205573 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.205579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.205587 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.205744 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.206105 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.206108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.206110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.206587 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:46.206777 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:46.207132 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 07:41:46.207438 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"spinlock_wait_cycles":34560,"thread_start_us":171,"threads_started":1}
I20260811 07:41:46.209290 3110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:41:46.214687 3110 catalog_manager.cc:1347] Generated new cluster ID: 153495ce3e3f4612b5f9d21ce36d3593
I20260811 07:41:46.214751 3110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:41:46.231097 3110 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:41:46.232232 3110 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:41:46.244462 3110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Generated new TSK 0
I20260811 07:41:46.245232 3110 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:41:46.282876 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:42967
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:46.400103 3123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:46.400398 3123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:46.400490 3123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:46.404505 3123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:46.404585 3123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:46.404671 3123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:41:46.410120 3123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:46.411195 3123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:46.412313 3123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:46.422130 3128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:46.422115 3129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:46.422115 3132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:46.422444 3123 server_base.cc:1034] running on GCE node
I20260811 07:41:46.422921 3123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:46.423576 3123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:46.424733 3123 hybrid_clock.cc:648] HybridClock initialized: now 1786434106424688 us; error 56 us; skew 500 ppm
Aug 11 07:41:46 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434106, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:46.427556 3123 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:41:46.428458 3123 webserver.cc:460] Webserver started at http://127.18.72.193:35971/ using document root <none> and password file <none>
I20260811 07:41:46.429008 3123 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:46.429064 3123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:46.429224 3123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:46.431164 3123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "2f2a890e7aee48dd94dc5fd89ae72d13"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "0a0b9169ac0a2617c603030f3b574573"
server_key_iv: "9c909d9d7b0bf045609be936d823d0a8"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.431706 3123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "2f2a890e7aee48dd94dc5fd89ae72d13"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "0a0b9169ac0a2617c603030f3b574573"
server_key_iv: "9c909d9d7b0bf045609be936d823d0a8"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.435384 3123 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:41:46.437757 3139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.438989 3123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:41:46.439126 3123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "2f2a890e7aee48dd94dc5fd89ae72d13"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "0a0b9169ac0a2617c603030f3b574573"
server_key_iv: "9c909d9d7b0bf045609be936d823d0a8"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.439239 3123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:46.455480 3123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:46.458323 3123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:46.458515 3123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:46.459136 3123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:46.460022 3123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:46.460089 3123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.460158 3123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:46.460210 3123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.469087 3123 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:37751
I20260811 07:41:46.469131 3255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:37751 every 8 connection(s)
I20260811 07:41:46.469954 3123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:41:46 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434106, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:46.479928 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 3123
I20260811 07:41:46.480082 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:41:46.480451 18723 external_mini_cluster.cc:1428] Setting key 2021bb4386200c3dec292925117d6f59
I20260811 07:41:46.483804 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.471738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:60181 (local address 127.18.72.254:42967)
0811 07:41:46.471882 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:46.471923 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:46.472746 (+ 823us) server_negotiation.cc:407] Connection header received
0811 07:41:46.474135 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:46.474139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:46.474195 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:46.474304 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:46.475440 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.476077 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.477196 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.477461 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.477575 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.479914 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:46.479918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:46.479921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:46.479954 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:46.481609 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.482252 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.482255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.482256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.482331 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.482729 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.482731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.482732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.482894 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:46.482986 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:46.483601 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 07:41:46.483694 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 07:41:46.484828 3259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.472169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:60181)
0811 07:41:46.472601 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:46.472642 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:46.473926 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:46.474480 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:46.474487 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:46.474823 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:46.475290 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.475302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.476202 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.476207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.477066 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.477075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.477613 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.477618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.477787 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.478322 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:46.478340 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:46.479720 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:46.481762 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:46.481771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:46.481788 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:46.482138 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:46.482433 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:46.482436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:46.482438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:46.482631 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:46.483015 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:46.483024 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:46.483362 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 07:41:46.483565 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 07:41:46.483928 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260811 07:41:46.486003 3256 heartbeater.cc:347] Connected to a master server at 127.18.72.254:42967
I20260811 07:41:46.486265 3256 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:46.486748 3256 heartbeater.cc:511] Master 127.18.72.254:42967 requested a full tablet report, sending...
I20260811 07:41:46.488327 3059 ts_manager.cc:194] Registered new tserver with Master: 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751)
I20260811 07:41:46.489413 3059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:60181
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:41:46.508371 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:42967
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:46.621289 3262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:46.621584 3262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:46.621661 3262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:46.625669 3262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:46.625761 3262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:46.625855 3262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:41:46.631068 3262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:46.632105 3262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:46.633159 3262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:46.643044 3268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:46.643054 3267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:46.643209 3262 server_base.cc:1034] running on GCE node
W20260811 07:41:46.643415 3270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:46.643862 3262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:46.644551 3262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:46.645710 3262 hybrid_clock.cc:648] HybridClock initialized: now 1786434106645687 us; error 36 us; skew 500 ppm
Aug 11 07:41:46 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434106, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:46.648603 3262 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:41:46.649557 3262 webserver.cc:460] Webserver started at http://127.18.72.194:34045/ using document root <none> and password file <none>
I20260811 07:41:46.650197 3262 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:46.650262 3262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:46.650496 3262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:46.652710 3262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "608699df4344e063c118e22e626bfa31"
server_key_iv: "9128e87b0ecf92f718eb0007d17b34e5"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.653326 3262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "608699df4344e063c118e22e626bfa31"
server_key_iv: "9128e87b0ecf92f718eb0007d17b34e5"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.657295 3262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 07:41:46.660112 3277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.661345 3262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:41:46.661479 3262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "608699df4344e063c118e22e626bfa31"
server_key_iv: "9128e87b0ecf92f718eb0007d17b34e5"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.661592 3262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:46.679440 3262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:46.682245 3262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:46.682449 3262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:46.683048 3262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:46.683859 3262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:46.683957 3262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.684032 3262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:46.684092 3262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.693110 3262 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:41721
I20260811 07:41:46.693136 3396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:41721 every 8 connection(s)
I20260811 07:41:46.693969 3262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:41:46.694751 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 3262
I20260811 07:41:46.694880 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:41:46.695174 18723 external_mini_cluster.cc:1428] Setting key 4aacb3f5696eca49eb32c8044841d01b
Aug 11 07:41:46 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434106, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:41:46.709162 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.696029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:55669 (local address 127.18.72.254:42967)
0811 07:41:46.696167 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:46.696171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:46.697615 (+ 1444us) server_negotiation.cc:407] Connection header received
0811 07:41:46.698527 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:46.698531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:46.698584 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:46.698682 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:46.699832 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.700415 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.702145 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.702385 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.702469 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.705464 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:46.705468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:46.705470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:46.705493 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:46.707130 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.707744 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.707748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.707750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.707837 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.708187 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.708190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.708191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.708372 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:46.708450 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:46.708946 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 07:41:46.709047 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:41:46.710047 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.696382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:55669)
0811 07:41:46.697468 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:46.697510 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:46.698323 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:46.698860 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:46.698867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:46.699208 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:46.699677 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.699690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.700544 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.700551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.702005 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.702020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.702553 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.702558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.702807 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.703447 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:46.703463 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:46.705297 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:46.707248 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:46.707254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:46.707269 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:46.707639 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:46.707965 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:46.707968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:46.707969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:46.708104 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:46.708463 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:46.708468 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:46.708730 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 07:41:46.708894 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 07:41:46.709129 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":715,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
I20260811 07:41:46.711233 3397 heartbeater.cc:347] Connected to a master server at 127.18.72.254:42967
I20260811 07:41:46.711515 3397 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:46.712157 3397 heartbeater.cc:511] Master 127.18.72.254:42967 requested a full tablet report, sending...
I20260811 07:41:46.713406 3059 ts_manager.cc:194] Registered new tserver with Master: bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
I20260811 07:41:46.714285 3059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:55669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:41:46.725965 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:42967
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:41:46.836464 3402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:41:46.836697 3402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:41:46.836766 3402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:41:46.840741 3402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:41:46.840794 3402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:41:46.840878 3402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:41:46.846115 3402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:41:46.847074 3402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:41:46.848146 3402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:41:46.857453 3411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:46.857554 3402 server_base.cc:1034] running on GCE node
W20260811 07:41:46.857463 3407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:41:46.857463 3409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:41:46.858263 3402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:41:46.858961 3402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:41:46.860140 3402 hybrid_clock.cc:648] HybridClock initialized: now 1786434106860127 us; error 23 us; skew 500 ppm
Aug 11 07:41:46 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434106, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:41:46.862768 3402 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:41:46.863587 3402 webserver.cc:460] Webserver started at http://127.18.72.195:33201/ using document root <none> and password file <none>
I20260811 07:41:46.864209 3402 fs_manager.cc:359] Metadata directory not provided
I20260811 07:41:46.864300 3402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:41:46.864506 3402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:41:46.866505 3402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "51a2e2c0c77f462a933b0e2da82dec19"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "caa04c2651166c58caa857e5d8f9d7c9"
server_key_iv: "f43526a9e1a3e70f9f118debd95a96ca"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.867074 3402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "51a2e2c0c77f462a933b0e2da82dec19"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "caa04c2651166c58caa857e5d8f9d7c9"
server_key_iv: "f43526a9e1a3e70f9f118debd95a96ca"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.870700 3402 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 07:41:46.873104 3418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.874298 3402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:41:46.874428 3402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "51a2e2c0c77f462a933b0e2da82dec19"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "caa04c2651166c58caa857e5d8f9d7c9"
server_key_iv: "f43526a9e1a3e70f9f118debd95a96ca"
server_key_version: "encryptionkey@0"
I20260811 07:41:46.874536 3402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:41:46.885723 3402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:41:46.888386 3402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:41:46.888576 3402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:41:46.889168 3402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:41:46.889964 3402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:41:46.890034 3402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.890102 3402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:41:46.890144 3402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:41:46.899101 3402 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:42223
I20260811 07:41:46.899127 3534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:42223 every 8 connection(s)
I20260811 07:41:46.899964 3402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:41:46.902064 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 3402
I20260811 07:41:46.902191 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:41:46.902546 18723 external_mini_cluster.cc:1428] Setting key e08a660c7b3c4672e0827dcff2d3fde3
Aug 11 07:41:46 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434106, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:41:46.913527 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.901673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:39231 (local address 127.18.72.254:42967)
0811 07:41:46.901824 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:46.901827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:46.902728 (+ 901us) server_negotiation.cc:407] Connection header received
0811 07:41:46.904487 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:46.904491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:46.904549 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:46.904631 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:46.905944 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.906545 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.907659 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.907944 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.908065 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.910281 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:46.910285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:46.910288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:46.910314 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:46.911521 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.912076 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.912079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.912081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.912163 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.912506 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.912510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.912512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.912685 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:46.912798 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:46.913252 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 07:41:46.913369 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:41:46.914268 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.902058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:39231)
0811 07:41:46.902561 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:46.902630 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:46.904265 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:46.904822 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:46.904828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:46.905343 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:41:46.905805 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.905818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.906669 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.906673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.907530 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.907539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.908100 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.908106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.908278 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.908762 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:41:46.908777 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:41:46.910124 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:41:46.911638 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:46.911644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:46.911658 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:46.911973 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:46.912262 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:41:46.912264 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:41:46.912266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:41:46.912408 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:41:46.912790 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:41:46.912796 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:41:46.913025 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 07:41:46.913210 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 07:41:46.913519 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":146,"threads_started":1}
I20260811 07:41:46.915511 3535 heartbeater.cc:347] Connected to a master server at 127.18.72.254:42967
I20260811 07:41:46.915774 3535 heartbeater.cc:464] Registering TS with master...
I20260811 07:41:46.916375 3535 heartbeater.cc:511] Master 127.18.72.254:42967 requested a full tablet report, sending...
I20260811 07:41:46.917330 3059 ts_manager.cc:194] Registered new tserver with Master: 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223)
I20260811 07:41:46.917874 3059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:39231
I20260811 07:41:46.927924 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:41:46.937438 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.929856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44046 (local address 127.18.72.254:42967)
0811 07:41:46.929997 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:46.930003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:46.930201 (+ 198us) server_negotiation.cc:407] Connection header received
0811 07:41:46.930345 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:46.930348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:46.930394 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:46.930477 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:41:46.931154 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.931645 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.932565 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.932774 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.932890 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.934242 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:41:46.934245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:41:46.934248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:41:46.934278 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:41:46.935692 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.936176 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.936180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.936181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.936265 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:41:46.936547 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:41:46.936551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:41:46.936552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:41:46.936758 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:41:46.936920 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:46.937166 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 07:41:46.937281 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:41:46.940302 3059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:41:46.943320 3059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:41:46.958173 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.952277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44502 (local address 127.18.72.195:42223)
0811 07:41:46.953092 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:46.953104 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:46.953128 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:41:46.953208 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:46.953214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:46.953419 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:46.953882 (+ 463us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:46.954040 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.955147 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.956821 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.957398 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.957527 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.957678 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:46.957912 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 07:41:46.958016 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":141,"threads_started":1}
I20260811 07:41:46.958209 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.951653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.18.72.193:37751)
0811 07:41:46.952706 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:46.952720 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:46.952746 (+ 26us) server_negotiation.cc:407] Connection header received
0811 07:41:46.952829 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:46.952835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:46.953239 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:46.953402 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:46.953778 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.954973 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.956519 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.957390 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.957530 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.957676 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:46.957965 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 07:41:46.958058 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":930,"thread_start_us":65,"threads_started":1}
I20260811 07:41:46.958209 3550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.951690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42223 (local address 127.0.0.1:44502)
0811 07:41:46.952519 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:46.952534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:46.952625 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:46.953572 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:46.953574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:46.953593 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:46.953742 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.953751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.955275 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.955278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.956674 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.956687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.957529 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.957533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.957653 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.957735 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 07:41:46.957858 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 07:41:46.958013 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":86,"thread_start_us":98,"threads_started":1}
I20260811 07:41:46.958227 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.951515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37751 (local address 127.0.0.1:45780)
0811 07:41:46.952043 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:46.952090 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:46.952219 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:46.953424 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:46.953432 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:46.953455 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:46.953620 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.953626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.955124 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.955127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.956397 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.956407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.957737 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.957745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.957799 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.957813 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 07:41:46.957914 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 07:41:46.958016 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":49,"thread_start_us":63,"threads_started":1}
I20260811 07:41:46.958918 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.952252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41721 (local address 127.0.0.1:54624)
0811 07:41:46.953002 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:46.953015 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:46.953118 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:46.953610 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:46.953613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:46.953631 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:46.953804 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.953810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.954778 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.954783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.956528 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:46.956541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.957590 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.957593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:46.958570 (+ 977us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.958629 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 07:41:46.958698 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 07:41:46.958765 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":132,"thread_start_us":111,"threads_started":1}
I20260811 07:41:46.959381 3552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:46.952575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54624 (local address 127.18.72.194:41721)
0811 07:41:46.952933 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:46.952947 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:46.953108 (+ 161us) server_negotiation.cc:407] Connection header received
0811 07:41:46.953227 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:46.953232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:46.953410 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:46.953549 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:46.953920 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.954646 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.956660 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:46.957430 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:46.958377 (+ 947us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:46.958529 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:46.959144 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 07:41:46.959224 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":128,"threads_started":1}
I20260811 07:41:46.961510 3189 tablet_service.cc:1467] Processing CreateTablet for tablet 757b55ee482c466a8a8acaff6d229924 (DEFAULT_TABLE table=test-table [id=70cfde3bc57e489992f9c152f4d0092d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:46.961582 3469 tablet_service.cc:1467] Processing CreateTablet for tablet 757b55ee482c466a8a8acaff6d229924 (DEFAULT_TABLE table=test-table [id=70cfde3bc57e489992f9c152f4d0092d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:46.961921 3331 tablet_service.cc:1467] Processing CreateTablet for tablet 757b55ee482c466a8a8acaff6d229924 (DEFAULT_TABLE table=test-table [id=70cfde3bc57e489992f9c152f4d0092d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:41:46.962622 3189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757b55ee482c466a8a8acaff6d229924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:46.962657 3469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757b55ee482c466a8a8acaff6d229924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:46.962777 3331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757b55ee482c466a8a8acaff6d229924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:41:46.969317 3553 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Bootstrap starting.
I20260811 07:41:46.970469 3555 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Bootstrap starting.
I20260811 07:41:46.972195 3554 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Bootstrap starting.
I20260811 07:41:46.972385 3553 tablet_bootstrap.cc:654] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:46.972651 3555 tablet_bootstrap.cc:654] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:46.974134 3555 log.cc:824] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:46.974133 3553 log.cc:824] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:46.974593 3554 tablet_bootstrap.cc:654] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Neither blocks nor log segments found. Creating new log.
I20260811 07:41:46.975365 3554 log.cc:824] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:41:46.976248 3553 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: No bootstrap required, opened a new log
I20260811 07:41:46.976495 3553 ts_tablet_manager.cc:1397] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 07:41:46.976869 3555 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: No bootstrap required, opened a new log
I20260811 07:41:46.977061 3555 ts_tablet_manager.cc:1397] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 07:41:46.977383 3554 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: No bootstrap required, opened a new log
I20260811 07:41:46.977581 3554 ts_tablet_manager.cc:1397] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:41:46.980631 3553 raft_consensus.cc:348] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:46.981022 3553 raft_consensus.cc:374] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:46.980865 3555 raft_consensus.cc:348] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:46.981128 3553 raft_consensus.cc:729] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2a890e7aee48dd94dc5fd89ae72d13, State: Initialized, Role: FOLLOWER
I20260811 07:41:46.981143 3555 raft_consensus.cc:374] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:46.981206 3555 raft_consensus.cc:729] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a2e2c0c77f462a933b0e2da82dec19, State: Initialized, Role: FOLLOWER
I20260811 07:41:46.981113 3554 raft_consensus.cc:348] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:46.981432 3554 raft_consensus.cc:374] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:41:46.981495 3554 raft_consensus.cc:729] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8ee4a46d8e4c81b946c4ac73b1c63e, State: Initialized, Role: FOLLOWER
I20260811 07:41:46.981683 3553 consensus_queue.cc:260] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:46.981683 3555 consensus_queue.cc:260] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:46.981945 3554 consensus_queue.cc:260] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:46.982597 3535 heartbeater.cc:503] Master 127.18.72.254:42967 was elected leader, sending a full tablet report...
I20260811 07:41:46.982630 3256 heartbeater.cc:503] Master 127.18.72.254:42967 was elected leader, sending a full tablet report...
I20260811 07:41:46.982736 3553 ts_tablet_manager.cc:1428] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:41:46.982955 3554 ts_tablet_manager.cc:1428] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:41:46.983269 3397 heartbeater.cc:503] Master 127.18.72.254:42967 was elected leader, sending a full tablet report...
I20260811 07:41:46.984115 3555 ts_tablet_manager.cc:1428] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 07:41:47.151000 3536 tablet.cc:2369] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:41:47.195984 3398 tablet.cc:2369] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:41:47.221266 3258 tablet.cc:2369] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:41:47.263579 3560 raft_consensus.cc:482] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:41:47.263788 3560 raft_consensus.cc:504] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:47.265046 3560 leader_election.cc:288] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751), bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
I20260811 07:41:47.269833 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:47.265286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:32787 (local address 127.18.72.193:37751)
0811 07:41:47.265425 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:47.265432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:47.265629 (+ 197us) server_negotiation.cc:407] Connection header received
0811 07:41:47.265689 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:47.265691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:47.265735 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:47.265844 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:47.266162 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.266926 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.268557 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.269152 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.269276 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:47.269331 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:47.269614 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 07:41:47.269686 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 07:41:47.269822 3552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:47.265422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:42629 (local address 127.18.72.194:41721)
0811 07:41:47.265554 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:47.265561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:47.265788 (+ 227us) server_negotiation.cc:407] Connection header received
0811 07:41:47.265913 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:47.265917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:47.265971 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:47.266023 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:41:47.266396 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.267103 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.268547 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.269149 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.269276 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:47.269336 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:47.269614 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 07:41:47.269686 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:41:47.269855 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:47.265217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37751 (local address 127.18.72.195:32787)
0811 07:41:47.265425 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:47.265441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:47.265550 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:47.265897 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:47.265900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:47.265917 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:47.266050 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:47.266056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.267047 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.267049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:47.268416 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:47.268427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.269277 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.269281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:47.269448 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:47.269509 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 07:41:47.269581 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:41:47.269704 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":98}
I20260811 07:41:47.269870 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:47.265302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41721 (local address 127.18.72.195:42629)
0811 07:41:47.265694 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:47.265704 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:41:47.265783 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:41:47.266164 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:41:47.266166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:41:47.266182 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:41:47.266278 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:47.266282 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.267201 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.267204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:47.268415 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:41:47.268427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.269277 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.269281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:41:47.269459 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:47.269502 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 07:41:47.269581 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 07:41:47.269696 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":88,"thread_start_us":128,"threads_started":1}
I20260811 07:41:47.270581 3210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757b55ee482c466a8a8acaff6d229924" candidate_uuid: "51a2e2c0c77f462a933b0e2da82dec19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" is_pre_election: true
I20260811 07:41:47.270578 3351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757b55ee482c466a8a8acaff6d229924" candidate_uuid: "51a2e2c0c77f462a933b0e2da82dec19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" is_pre_election: true
I20260811 07:41:47.270897 3210 raft_consensus.cc:2454] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a2e2c0c77f462a933b0e2da82dec19 in term 0.
I20260811 07:41:47.270898 3351 raft_consensus.cc:2454] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a2e2c0c77f462a933b0e2da82dec19 in term 0.
I20260811 07:41:47.271421 3420 leader_election.cc:302] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13, 51a2e2c0c77f462a933b0e2da82dec19, bb8ee4a46d8e4c81b946c4ac73b1c63e; no voters:
I20260811 07:41:47.271724 3560 raft_consensus.cc:2781] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:41:47.271817 3560 raft_consensus.cc:482] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:41:47.271850 3560 raft_consensus.cc:3037] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:47.272918 3560 raft_consensus.cc:504] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:47.273264 3560 leader_election.cc:288] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [CANDIDATE]: Term 1 election: Requested vote from peers 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751), bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
I20260811 07:41:47.273607 3210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757b55ee482c466a8a8acaff6d229924" candidate_uuid: "51a2e2c0c77f462a933b0e2da82dec19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13"
I20260811 07:41:47.273667 3351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757b55ee482c466a8a8acaff6d229924" candidate_uuid: "51a2e2c0c77f462a933b0e2da82dec19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e"
I20260811 07:41:47.273752 3210 raft_consensus.cc:3037] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:47.273847 3351 raft_consensus.cc:3037] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:41:47.274822 3210 raft_consensus.cc:2454] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a2e2c0c77f462a933b0e2da82dec19 in term 1.
I20260811 07:41:47.274962 3351 raft_consensus.cc:2454] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a2e2c0c77f462a933b0e2da82dec19 in term 1.
I20260811 07:41:47.275162 3422 leader_election.cc:302] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13, 51a2e2c0c77f462a933b0e2da82dec19; no voters:
I20260811 07:41:47.275353 3560 raft_consensus.cc:2781] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:41:47.275599 3560 raft_consensus.cc:686] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 LEADER]: Becoming Leader. State: Replica: 51a2e2c0c77f462a933b0e2da82dec19, State: Running, Role: LEADER
I20260811 07:41:47.275986 3560 consensus_queue.cc:237] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:41:47.279238 3059 catalog_manager.cc:5495] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195). New cstate: current_term: 1 leader_uuid: "51a2e2c0c77f462a933b0e2da82dec19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } health_report { overall_health: UNKNOWN } } }
I20260811 07:41:47.318771 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:47.314951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.18.72.195:42223)
0811 07:41:47.315110 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 07:41:47.315115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:41:47.315131 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:41:47.315273 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:41:47.315276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:41:47.315336 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:41:47.315457 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:41:47.315736 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.316278 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.317158 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:41:47.317404 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:41:47.317520 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:41:47.317877 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:41:47.318341 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:41:47.318486 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 07:41:47.318532 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 07:41:47.318596 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:41:47.324978 3351 raft_consensus.cc:1261] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 FOLLOWER]: Refusing update from remote peer 51a2e2c0c77f462a933b0e2da82dec19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:47.324980 3210 raft_consensus.cc:1261] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Refusing update from remote peer 51a2e2c0c77f462a933b0e2da82dec19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:41:47.325750 3563 consensus_queue.cc:1035] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:47.326002 3560 consensus_queue.cc:1035] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:41:47.333570 3566 mvcc.cc:205] Tried to move back new op lower bound from 7317234103594930176 to 7317234103404994560. Current Snapshot: MvccSnapshot[applied={T|T < 7317234103594930176}]
I20260811 07:41:47.335665 3568 mvcc.cc:205] Tried to move back new op lower bound from 7317234103594930176 to 7317234103404994560. Current Snapshot: MvccSnapshot[applied={T|T < 7317234103594930176}]
I20260811 07:41:47.335843 3567 mvcc.cc:205] Tried to move back new op lower bound from 7317234103594930176 to 7317234103404994560. Current Snapshot: MvccSnapshot[applied={T|T < 7317234103594930176}]
I20260811 07:41:47.343161 3057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:41:47.343345 3057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:41:47.345865 3057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Sending DeleteTablet for 3 replicas of tablet 757b55ee482c466a8a8acaff6d229924
I20260811 07:41:47.346621 3469 tablet_service.cc:1514] Processing DeleteTablet for tablet 757b55ee482c466a8a8acaff6d229924 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:44502
I20260811 07:41:47.346654 3189 tablet_service.cc:1514] Processing DeleteTablet for tablet 757b55ee482c466a8a8acaff6d229924 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45780
I20260811 07:41:47.347127 3577 tablet_replica.cc:330] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: stopping tablet replica
I20260811 07:41:47.347117 3331 tablet_service.cc:1514] Processing DeleteTablet for tablet 757b55ee482c466a8a8acaff6d229924 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:54624
I20260811 07:41:47.347466 3577 raft_consensus.cc:2229] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:41:47.347613 3578 tablet_replica.cc:330] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: stopping tablet replica
I20260811 07:41:47.347867 3577 raft_consensus.cc:2258] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:47.347966 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 3123
I20260811 07:41:47.348037 3578 raft_consensus.cc:2229] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:41:47.348357 3578 raft_consensus.cc:2258] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:41:47.349397 3577 ts_tablet_manager.cc:1905] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:41:47.350310 3578 ts_tablet_manager.cc:1905] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:41:47.351567 3577 ts_tablet_manager.cc:1918] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:47.351636 3577 log.cc:1196] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/757b55ee482c466a8a8acaff6d229924
I20260811 07:41:47.351905 3577 ts_tablet_manager.cc:1939] T 757b55ee482c466a8a8acaff6d229924 P 51a2e2c0c77f462a933b0e2da82dec19: Deleting consensus metadata
I20260811 07:41:47.353183 3043 catalog_manager.cc:4841] TS 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223): tablet 757b55ee482c466a8a8acaff6d229924 (table test-table [id=70cfde3bc57e489992f9c152f4d0092d]) successfully deleted
I20260811 07:41:47.354481 3578 ts_tablet_manager.cc:1918] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:41:47.354595 3578 log.cc:1196] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/757b55ee482c466a8a8acaff6d229924
I20260811 07:41:47.354983 3578 ts_tablet_manager.cc:1939] T 757b55ee482c466a8a8acaff6d229924 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Deleting consensus metadata
I20260811 07:41:47.356469 3044 catalog_manager.cc:4841] TS bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721): tablet 757b55ee482c466a8a8acaff6d229924 (table test-table [id=70cfde3bc57e489992f9c152f4d0092d]) successfully deleted
I20260811 07:41:47.365875 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 3262
W20260811 07:41:47.365869 3046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:37751 (error 108)
I20260811 07:41:47.370273 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:41:47.369891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37751 (local address 127.0.0.1:45788)
0811 07:41:47.370071 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 07:41:47.370160 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 07:41:47.370563 3046 catalog_manager.cc:4584] TS 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 757b55ee482c466a8a8acaff6d229924: Network error: Client connection negotiation failed: client connection to 127.18.72.193:37751: connect: Connection refused (error 111)
I20260811 07:41:47.375546 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 3402
I20260811 07:41:47.384944 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 3024
I20260811 07:42:07.395191 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:42967
--webserver_interface=127.18.72.254
--webserver_port=41963
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:07.515606 3581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:07.515898 3581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:07.515995 3581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:07.519941 3581 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 07:42:07.520025 3581 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 07:42:07.520052 3581 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 07:42:07.520088 3581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:07.520130 3581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:07.520171 3581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:07.520205 3581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:07.520236 3581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:07.525372 3581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:42967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=41963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:07.526387 3581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:07.527415 3581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:42:07.542032 3581 server_base.cc:1034] running on GCE node
W20260811 07:42:07.541968 3590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:07.541960 3588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:07.541967 3587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:07.542789 3581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:07.543974 3581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:07.545173 3581 hybrid_clock.cc:648] HybridClock initialized: now 1786434127545154 us; error 42 us; skew 500 ppm
Aug 11 07:42:07 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434127, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:07.553745 3581 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:42:07.554667 3581 webserver.cc:460] Webserver started at http://127.18.72.254:41963/ using document root <none> and password file <none>
I20260811 07:42:07.555270 3581 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:07.555377 3581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:07.559302 3581 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:07.561501 3597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:07.562708 3581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:42:07.562867 3581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "c14340c2c2f14e1c8ca60163e8233673"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "f2d12dfd9575765feada411b44030ade"
server_key_iv: "aaf98d7971dafc863c0e95df1106b5bc"
server_key_version: "encryptionkey@0"
I20260811 07:42:07.563457 3581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:07.617259 3581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:07.620467 3581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:07.620676 3581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:07.628273 3581 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:42967
I20260811 07:42:07.628302 3652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:42967 every 8 connection(s)
I20260811 07:42:07.629528 3581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:07.633373 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 3581
I20260811 07:42:07.633824 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:37751
--local_ip_for_outbound_sockets=127.18.72.193
--tserver_master_addrs=127.18.72.254:42967
--webserver_port=35971
--webserver_interface=127.18.72.193
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:42:07.634260 3653 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:42:07.636222 3653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Bootstrap starting.
I20260811 07:42:07.661587 3653 log.cc:824] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:07.673151 3653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:42:07.673593 3653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Bootstrap complete.
I20260811 07:42:07.679556 3653 raft_consensus.cc:348] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:42:07.680193 3653 raft_consensus.cc:729] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14340c2c2f14e1c8ca60163e8233673, State: Initialized, Role: FOLLOWER
I20260811 07:42:07.680759 3653 consensus_queue.cc:260] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:42:07.680915 3653 raft_consensus.cc:388] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:07.680994 3653 raft_consensus.cc:482] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:07.681093 3653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:42:07.683641 3653 raft_consensus.cc:504] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:42:07.684038 3653 leader_election.cc:302] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c14340c2c2f14e1c8ca60163e8233673; no voters:
I20260811 07:42:07.684382 3653 leader_election.cc:288] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:42:07.684510 3657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:42:07.684795 3657 raft_consensus.cc:686] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 2 LEADER]: Becoming Leader. State: Replica: c14340c2c2f14e1c8ca60163e8233673, State: Running, Role: LEADER
I20260811 07:42:07.685092 3657 consensus_queue.cc:237] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:42:07.685680 3653 sys_catalog.cc:564] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:07.686026 3661 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c14340c2c2f14e1c8ca60163e8233673. Latest consensus state: current_term: 2 leader_uuid: "c14340c2c2f14e1c8ca60163e8233673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } } }
I20260811 07:42:07.686157 3661 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:07.686421 3659 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c14340c2c2f14e1c8ca60163e8233673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } } }
I20260811 07:42:07.686501 3659 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:07.686581 3663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:07.691309 3663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:07.692091 3663 catalog_manager.cc:1259] Loaded cluster ID: 153495ce3e3f4612b5f9d21ce36d3593
I20260811 07:42:07.692157 3663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:07.694559 3663 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:07.695242 3663 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Loaded TSK: 0
I20260811 07:42:07.718763 3663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Generated new TSK 1
I20260811 07:42:07.719024 3663 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:42:07.767035 3655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:07.767313 3655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:07.767421 3655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:07.771462 3655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:07.771512 3655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:07.771591 3655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:42:07.776691 3655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:37751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=35971
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:07.777644 3655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:07.778644 3655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:07.789206 3683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:07.789193 3682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:07.789390 3685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:07.789983 3655 server_base.cc:1034] running on GCE node
I20260811 07:42:07.790448 3655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:07.791185 3655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:07.792380 3655 hybrid_clock.cc:648] HybridClock initialized: now 1786434127792353 us; error 32 us; skew 500 ppm
Aug 11 07:42:07 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434127, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:07.795718 3655 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:42:07.796660 3655 webserver.cc:460] Webserver started at http://127.18.72.193:35971/ using document root <none> and password file <none>
I20260811 07:42:07.797261 3655 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:07.797372 3655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:07.801339 3655 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 07:42:07.803617 3692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:07.804914 3655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:42:07.805051 3655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "2f2a890e7aee48dd94dc5fd89ae72d13"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "0a0b9169ac0a2617c603030f3b574573"
server_key_iv: "9c909d9d7b0bf045609be936d823d0a8"
server_key_version: "encryptionkey@0"
I20260811 07:42:07.805614 3655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:07.827548 3655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:07.831173 3655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:07.831403 3655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:07.832155 3655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:07.833395 3700 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 07:42:07.837664 3655 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:42:07.837733 3655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 07:42:07.837781 3655 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:42:07.839437 3655 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:42:07.839496 3655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 07:42:07.839781 3700 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Bootstrap starting.
I20260811 07:42:07.851235 3655 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:37751
I20260811 07:42:07.851318 3814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:37751 every 8 connection(s)
I20260811 07:42:07.852591 3655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:42:07.860913 3700 log.cc:824] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:07.862378 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 3655
I20260811 07:42:07.862879 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:41721
--local_ip_for_outbound_sockets=127.18.72.194
--tserver_master_addrs=127.18.72.254:42967
--webserver_port=34045
--webserver_interface=127.18.72.194
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:42:07 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434127, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:07.869738 3700 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:42:07.870170 3700 tablet_bootstrap.cc:492] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Bootstrap complete.
I20260811 07:42:07.870326 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:07.854710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:34927)
0811 07:42:07.855278 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:07.855319 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:07.856177 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:07.857260 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:07.857269 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:07.857686 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:07.858154 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:07.858168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:07.859484 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:07.859487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:07.860403 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:07.860413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:07.861160 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:07.861163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:07.861348 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:07.862027 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:07.862047 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:07.864021 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:07.866536 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:07.866545 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:07.866561 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:07.866936 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:07.867396 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:07.867400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:07.867402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:07.867578 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:07.868430 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:07.868436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:07.868703 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 07:42:07.868897 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 07:42:07.869210 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":182,"threads_started":1}
I20260811 07:42:07.870445 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:07.854676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:34927 (local address 127.18.72.254:42967)
0811 07:42:07.855073 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:07.855095 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:07.855440 (+ 345us) server_negotiation.cc:407] Connection header received
0811 07:42:07.856590 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:07.856600 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:07.856887 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:07.857210 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:07.858333 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:07.859306 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:07.860529 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:07.861010 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:07.861204 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:07.864254 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:07.864261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:07.864278 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:07.864311 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:07.866419 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:07.867085 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:07.867094 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:07.867103 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:07.867279 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:07.867696 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:07.867702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:07.867707 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:07.868300 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:07.868448 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:07.869093 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 07:42:07.869466 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":134,"threads_started":1}
I20260811 07:42:07.871692 3700 ts_tablet_manager.cc:1397] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.008s
I20260811 07:42:07.873441 3815 heartbeater.cc:347] Connected to a master server at 127.18.72.254:42967
I20260811 07:42:07.873680 3815 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:07.874240 3815 heartbeater.cc:511] Master 127.18.72.254:42967 requested a full tablet report, sending...
I20260811 07:42:07.874413 3700 raft_consensus.cc:348] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:42:07.874888 3700 raft_consensus.cc:729] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2a890e7aee48dd94dc5fd89ae72d13, State: Initialized, Role: FOLLOWER
I20260811 07:42:07.875334 3700 consensus_queue.cc:260] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } } peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } }
I20260811 07:42:07.876401 3700 ts_tablet_manager.cc:1428] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260811 07:42:07.878285 3614 ts_manager.cc:194] Registered new tserver with Master: 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751)
I20260811 07:42:07.881794 3614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:34927
I20260811 07:42:07.884507 3815 heartbeater.cc:503] Master 127.18.72.254:42967 was elected leader, sending a full tablet report...
Aug 11 07:42:07 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434127, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.193@KRBTEST.COM
I20260811 07:42:07.892138 3822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:07.881131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37751 (local address 127.0.0.1:38370)
0811 07:42:07.881568 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:07.881618 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:07.881746 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:07.882383 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:07.882391 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:07.882836 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:07.882972 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:07.882980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:07.884331 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:07.884335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:07.885309 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:07.885320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:07.886231 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:07.886234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:07.886313 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:07.886910 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:07.886923 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:07.888416 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:07.890336 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:07.890345 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:07.890352 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:07.890579 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:07.890889 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:07.890894 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:07.890896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:07.890982 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:07.891440 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:07.891447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:07.891574 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 07:42:07.891713 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 07:42:07.891953 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":125,"threads_started":1}
I20260811 07:42:07.892138 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:07.881220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38370 (local address 127.18.72.193:37751)
0811 07:42:07.881912 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:07.881925 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:07.881949 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:42:07.882018 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:07.882023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:07.882220 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:07.882362 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:07.883106 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:07.884211 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:07.885468 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:07.886010 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:07.886112 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:07.888555 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:07.888565 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:07.888578 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:07.888609 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:07.890210 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:07.890698 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:07.890704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:07.890705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:07.890788 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:07.891076 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:07.891081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:07.891082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:07.891317 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:07.891468 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:07.891808 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 07:42:07.891996 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":129,"threads_started":1}
I20260811 07:42:07.893162 3745 tablet_service.cc:1514] Processing DeleteTablet for tablet 757b55ee482c466a8a8acaff6d229924 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:41:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:38370
I20260811 07:42:07.894155 3824 tablet_replica.cc:330] stopping tablet replica
I20260811 07:42:07.894457 3824 raft_consensus.cc:2229] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:07.894708 3824 raft_consensus.cc:2258] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:07.895663 3824 ts_tablet_manager.cc:1905] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:07.899174 3824 ts_tablet_manager.cc:1918] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:07.899302 3824 log.cc:1196] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal/wals/757b55ee482c466a8a8acaff6d229924
I20260811 07:42:07.899575 3824 ts_tablet_manager.cc:1939] T 757b55ee482c466a8a8acaff6d229924 P 2f2a890e7aee48dd94dc5fd89ae72d13: Deleting consensus metadata
I20260811 07:42:07.901229 3602 catalog_manager.cc:4841] TS 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751): tablet 757b55ee482c466a8a8acaff6d229924 (table test-table [id=70cfde3bc57e489992f9c152f4d0092d]) successfully deleted
W20260811 07:42:07.983618 3820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:07.983875 3820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:07.983930 3820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:07.988202 3820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:07.988255 3820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:07.988310 3820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:42:07.993402 3820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:41721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=34045
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:07.994323 3820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:07.995280 3820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:08.004617 3832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:08.004709 3829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:08.004621 3830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:08.005003 3820 server_base.cc:1034] running on GCE node
I20260811 07:42:08.005426 3820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:08.006035 3820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:08.007233 3820 hybrid_clock.cc:648] HybridClock initialized: now 1786434128007224 us; error 29 us; skew 500 ppm
Aug 11 07:42:08 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434128, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:08.010167 3820 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:42:08.011088 3820 webserver.cc:460] Webserver started at http://127.18.72.194:34045/ using document root <none> and password file <none>
I20260811 07:42:08.011654 3820 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:08.011776 3820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:08.015571 3820 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:08.017686 3840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:08.018934 3820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 07:42:08.019085 3820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "608699df4344e063c118e22e626bfa31"
server_key_iv: "9128e87b0ecf92f718eb0007d17b34e5"
server_key_version: "encryptionkey@0"
I20260811 07:42:08.019626 3820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:08.036013 3820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:08.038779 3820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:08.039007 3820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:08.039592 3820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:08.040407 3820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:08.040478 3820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:08.040547 3820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:08.040596 3820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:08.050026 3820 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:41721
I20260811 07:42:08.050055 3959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:41721 every 8 connection(s)
I20260811 07:42:08.051016 3820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:42:08.051251 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 3820
I20260811 07:42:08.051785 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:42223
--local_ip_for_outbound_sockets=127.18.72.195
--tserver_master_addrs=127.18.72.254:42967
--webserver_port=33201
--webserver_interface=127.18.72.195
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:42:08 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434128, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:08.067991 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.053011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:45941 (local address 127.18.72.254:42967)
0811 07:42:08.053154 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.053158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.054332 (+ 1174us) server_negotiation.cc:407] Connection header received
0811 07:42:08.056399 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.056402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.056454 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.056526 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:08.058233 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.058748 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.060686 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.060918 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.061024 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.063874 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:08.063880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:08.063885 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:08.063958 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:08.065707 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:08.066415 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:08.066419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:08.066421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:08.066518 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:08.066839 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:08.066843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:08.066844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:08.067080 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:08.067177 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.067646 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.067790 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 07:42:08.068687 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.053479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:45941)
0811 07:42:08.054130 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.054193 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:08.056148 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:08.056721 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:08.056733 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:08.057320 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:08.058063 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.058088 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.058887 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.058892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.060541 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.060555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.061083 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.061091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.061342 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.062015 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:08.062043 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:08.063698 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:08.065837 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:08.065844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:08.065860 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:08.066266 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:08.066614 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:08.066616 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:08.066617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:08.066747 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:08.067171 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:08.067175 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:08.067393 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 07:42:08.067582 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 07:42:08.067851 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":191,"threads_started":1}
I20260811 07:42:08.069736 3961 heartbeater.cc:347] Connected to a master server at 127.18.72.254:42967
I20260811 07:42:08.069986 3961 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:08.070451 3961 heartbeater.cc:511] Master 127.18.72.254:42967 requested a full tablet report, sending...
I20260811 07:42:08.071499 3614 ts_manager.cc:194] Registered new tserver with Master: bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
I20260811 07:42:08.072093 3614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:45941
W20260811 07:42:08.184171 3963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:08.184437 3963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:08.184520 3963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:08.188964 3963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:08.189044 3963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:08.189127 3963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:42:08.194226 3963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:42223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=33201
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:08.195216 3963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:08.196273 3963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:08.205549 3970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:08.205608 3972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:08.205751 3963 server_base.cc:1034] running on GCE node
W20260811 07:42:08.205617 3969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:08.206332 3963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:08.206934 3963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:08.208117 3963 hybrid_clock.cc:648] HybridClock initialized: now 1786434128208087 us; error 40 us; skew 500 ppm
Aug 11 07:42:08 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434128, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:08.210872 3963 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:42:08.211642 3963 webserver.cc:460] Webserver started at http://127.18.72.195:33201/ using document root <none> and password file <none>
I20260811 07:42:08.212206 3963 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:08.212288 3963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:08.215811 3963 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:08.217836 3979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:08.219161 3963 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:08.219290 3963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "51a2e2c0c77f462a933b0e2da82dec19"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "caa04c2651166c58caa857e5d8f9d7c9"
server_key_iv: "f43526a9e1a3e70f9f118debd95a96ca"
server_key_version: "encryptionkey@0"
I20260811 07:42:08.219836 3963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:08.230645 3963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:08.233757 3963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:08.233943 3963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:08.234488 3963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:08.235280 3963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:08.235348 3963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:08.235414 3963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:08.235466 3963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:08.244536 3963 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:42223
I20260811 07:42:08.244585 4099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:42223 every 8 connection(s)
I20260811 07:42:08.245508 3963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:42:08.251228 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 3963
Aug 11 07:42:08 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434128, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:08.260236 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.247340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:33917 (local address 127.18.72.254:42967)
0811 07:42:08.247497 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.247500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.248338 (+ 838us) server_negotiation.cc:407] Connection header received
0811 07:42:08.249620 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.249623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.249671 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.249758 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:08.250859 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.251441 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.252988 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.253214 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.253325 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.256085 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:08.256095 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:08.256097 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:08.256119 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:08.257662 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:08.258251 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:08.258254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:08.258256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:08.258347 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:08.258690 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:08.258694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:08.258696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:08.258935 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:08.259040 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.259944 (+ 904us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.260061 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":58}
I20260811 07:42:08.260567 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.251947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.18.72.254:42967)
0811 07:42:08.252188 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.252193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.252381 (+ 188us) server_negotiation.cc:407] Connection header received
0811 07:42:08.252481 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.252484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.252526 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.252608 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:08.253316 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.253878 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.255095 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.255469 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.255579 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.257222 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:08.257228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:08.257230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:08.257271 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:08.258948 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:08.259312 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:08.259315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:08.259316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:08.259394 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:08.259678 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:08.259680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:08.259681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:08.259876 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:08.260101 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.260256 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.260388 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":79,"threads_started":1}
I20260811 07:42:08.261188 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.247705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:33917)
0811 07:42:08.248180 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.248222 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:08.249418 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:08.249927 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:08.249933 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:08.250273 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:08.250727 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.250740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.251564 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.251568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.252870 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.252880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.253363 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.253367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.253554 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.254160 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:08.254180 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:08.255875 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:08.257793 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:08.257801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:08.257831 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:08.258119 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:08.258446 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:08.258448 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:08.258449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:08.258580 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:08.259341 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:08.259346 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:08.259633 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 07:42:08.259814 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 07:42:08.260160 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":148,"threads_started":1}
I20260811 07:42:08.262315 4100 heartbeater.cc:347] Connected to a master server at 127.18.72.254:42967
I20260811 07:42:08.262559 4100 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:08.263023 4100 heartbeater.cc:511] Master 127.18.72.254:42967 requested a full tablet report, sending...
I20260811 07:42:08.264045 3614 ts_manager.cc:194] Registered new tserver with Master: 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223)
I20260811 07:42:08.264585 3614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:33917
I20260811 07:42:08.273820 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:42:08.277486 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.274387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50306 (local address 127.18.72.254:42967)
0811 07:42:08.274504 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.274508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.274695 (+ 187us) server_negotiation.cc:407] Connection header received
0811 07:42:08.274815 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.274817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.274901 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.274979 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:42:08.275159 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.275626 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.276525 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.276731 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.276808 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.277049 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:42:08.277162 (+ 113us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 07:42:08.277350 (+ 188us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50306: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 07:42:08.277571 4103 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50306: token expired
W20260811 07:42:08.277915 3540 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.72.254:42967)
I20260811 07:42:08.284405 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.278322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50310 (local address 127.18.72.254:42967)
0811 07:42:08.278430 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.278433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.278445 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:42:08.278508 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.278509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.278538 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.278597 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:08.279147 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.279584 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.280387 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.280578 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.280652 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.281799 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:08.281802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:08.281804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:08.281824 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:08.283030 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:08.283392 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:08.283394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:08.283395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:08.283461 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:08.283680 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:08.283682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:08.283683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:08.283820 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:08.283968 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.284133 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.284216 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:42:08.286608 3614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:42:08.290387 3614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:42:08.298532 3745 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2282181bd54d7e9175129504690b99 (DEFAULT_TABLE table=test-table [id=69d609876c9d4ca998a606f5ce1f4e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:08.299348 3745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2282181bd54d7e9175129504690b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:08.302412 3700 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Bootstrap starting.
I20260811 07:42:08.303078 3822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.297131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42223 (local address 127.0.0.1:45260)
0811 07:42:08.297278 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.297294 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:08.297376 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:08.298312 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:08.298315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:08.298335 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:08.298440 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.298446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.300247 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.300251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.301470 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.301483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.302623 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.302627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.302723 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.302787 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 07:42:08.302862 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 07:42:08.302941 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 07:42:08.303121 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.297359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45260 (local address 127.18.72.195:42223)
0811 07:42:08.297712 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.297726 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.297750 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:42:08.297836 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.297859 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.298094 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.298313 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:08.298541 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.300119 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.301623 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.302513 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.302621 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.302765 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.302890 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.302981 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":143,"threads_started":1}
I20260811 07:42:08.303319 3700 tablet_bootstrap.cc:654] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:08.303664 4110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.297421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41721 (local address 127.0.0.1:39994)
0811 07:42:08.297691 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.297705 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:08.297804 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:08.299006 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:08.299010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:08.299028 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:08.299126 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.299132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.300592 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.300598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.301399 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.301408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.303168 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.303172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.303295 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.303326 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 07:42:08.303407 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 07:42:08.303483 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":6,"thread_start_us":134,"threads_started":1}
I20260811 07:42:08.303822 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.297906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39994 (local address 127.18.72.194:41721)
0811 07:42:08.298291 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.298304 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.298325 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:42:08.298415 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.298420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.298636 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.298803 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:08.299255 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.300479 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.301718 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.302697 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.302810 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.302956 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.303469 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.303627 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":167,"threads_started":1}
I20260811 07:42:08.305080 3700 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: No bootstrap required, opened a new log
I20260811 07:42:08.305181 3700 ts_tablet_manager.cc:1397] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 07:42:08.305639 3700 raft_consensus.cc:348] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.305768 3700 raft_consensus.cc:374] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:08.305810 3700 raft_consensus.cc:729] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2a890e7aee48dd94dc5fd89ae72d13, State: Initialized, Role: FOLLOWER
I20260811 07:42:08.306093 3700 consensus_queue.cc:260] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.306464 3700 ts_tablet_manager.cc:1428] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 07:42:08.306433 4031 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2282181bd54d7e9175129504690b99 (DEFAULT_TABLE table=test-table [id=69d609876c9d4ca998a606f5ce1f4e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:08.307512 4031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2282181bd54d7e9175129504690b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:08.307400 3889 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2282181bd54d7e9175129504690b99 (DEFAULT_TABLE table=test-table [id=69d609876c9d4ca998a606f5ce1f4e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:08.308440 3889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2282181bd54d7e9175129504690b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:08.314358 4113 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Bootstrap starting.
I20260811 07:42:08.314571 4114 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Bootstrap starting.
I20260811 07:42:08.316690 4113 tablet_bootstrap.cc:654] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:08.317091 4114 tablet_bootstrap.cc:654] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:08.317783 4113 log.cc:824] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:08.317792 4114 log.cc:824] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:08.319705 4114 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: No bootstrap required, opened a new log
I20260811 07:42:08.319757 4113 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: No bootstrap required, opened a new log
I20260811 07:42:08.319962 4114 ts_tablet_manager.cc:1397] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:42:08.319990 4113 ts_tablet_manager.cc:1397] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:42:08.322716 4114 raft_consensus.cc:348] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.322986 4114 raft_consensus.cc:374] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:08.322772 4113 raft_consensus.cc:348] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.323053 4114 raft_consensus.cc:729] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8ee4a46d8e4c81b946c4ac73b1c63e, State: Initialized, Role: FOLLOWER
I20260811 07:42:08.323104 4113 raft_consensus.cc:374] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:08.323165 4113 raft_consensus.cc:729] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a2e2c0c77f462a933b0e2da82dec19, State: Initialized, Role: FOLLOWER
I20260811 07:42:08.323496 4114 consensus_queue.cc:260] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.323609 4113 consensus_queue.cc:260] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.324570 4114 ts_tablet_manager.cc:1428] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:42:08.324589 4113 ts_tablet_manager.cc:1428] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 07:42:08.324659 4100 heartbeater.cc:503] Master 127.18.72.254:42967 was elected leader, sending a full tablet report...
I20260811 07:42:08.324918 3961 heartbeater.cc:503] Master 127.18.72.254:42967 was elected leader, sending a full tablet report...
W20260811 07:42:08.353814 3816 tablet.cc:2369] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:42:08.496469 4101 tablet.cc:2369] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:42:08.552235 3962 tablet.cc:2369] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:42:08.647691 4117 raft_consensus.cc:482] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:08.648038 4117 raft_consensus.cc:504] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.649430 4117 leader_election.cc:288] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223), 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751)
I20260811 07:42:08.653522 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.649802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:59809 (local address 127.18.72.195:42223)
0811 07:42:08.650010 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.650015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.650203 (+ 188us) server_negotiation.cc:407] Connection header received
0811 07:42:08.650380 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.650384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.650426 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.650512 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:08.650843 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.651333 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.652475 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.652870 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.652998 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.653054 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.653320 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.653389 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 07:42:08.653568 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.649856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:49657 (local address 127.18.72.193:37751)
0811 07:42:08.650092 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.650097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.650112 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:42:08.650294 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.650297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.650361 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.650478 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:08.650810 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.651331 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.652508 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.652947 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.653082 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.653236 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.653356 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.653438 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":62,"threads_started":1}
I20260811 07:42:08.653573 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.649694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42223 (local address 127.18.72.194:59809)
0811 07:42:08.650102 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.650113 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:08.650242 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:08.650525 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:08.650528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:08.650568 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:08.650724 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.650729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.651455 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.651460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.652346 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.652356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.652998 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.653002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.653161 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.653182 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 07:42:08.653303 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 07:42:08.653375 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":99,"spinlock_wait_cycles":768,"thread_start_us":102,"threads_started":1}
I20260811 07:42:08.653581 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.649690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37751 (local address 127.18.72.194:49657)
0811 07:42:08.650024 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.650041 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:08.650150 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:08.650489 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:08.650492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:08.650509 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:08.650684 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.650689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.651461 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.651467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.652379 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.652389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.653075 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.653078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.653218 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.653242 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 07:42:08.653324 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 07:42:08.653397 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":84,"spinlock_wait_cycles":18944,"thread_start_us":70,"threads_started":1}
I20260811 07:42:08.654215 3767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2282181bd54d7e9175129504690b99" candidate_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" is_pre_election: true
I20260811 07:42:08.654562 3767 raft_consensus.cc:2454] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb8ee4a46d8e4c81b946c4ac73b1c63e in term 0.
I20260811 07:42:08.654625 4118 raft_consensus.cc:482] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:08.654707 4052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2282181bd54d7e9175129504690b99" candidate_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a2e2c0c77f462a933b0e2da82dec19" is_pre_election: true
I20260811 07:42:08.654834 4118 raft_consensus.cc:504] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.655076 3844 leader_election.cc:302] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13, bb8ee4a46d8e4c81b946c4ac73b1c63e; no voters:
I20260811 07:42:08.655210 4052 raft_consensus.cc:2454] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb8ee4a46d8e4c81b946c4ac73b1c63e in term 0.
I20260811 07:42:08.655385 4117 raft_consensus.cc:2781] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:42:08.655472 4117 raft_consensus.cc:482] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:42:08.655515 4117 raft_consensus.cc:3037] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:08.656046 4118 leader_election.cc:288] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721), 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751)
I20260811 07:42:08.657006 4117 raft_consensus.cc:504] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.657401 4117 leader_election.cc:288] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [CANDIDATE]: Term 1 election: Requested vote from peers 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223), 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751)
I20260811 07:42:08.657671 4052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2282181bd54d7e9175129504690b99" candidate_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a2e2c0c77f462a933b0e2da82dec19"
I20260811 07:42:08.657779 4052 raft_consensus.cc:3037] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:08.657768 3767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2282181bd54d7e9175129504690b99" candidate_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13"
I20260811 07:42:08.657902 3767 raft_consensus.cc:3037] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:08.659078 4052 raft_consensus.cc:2454] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb8ee4a46d8e4c81b946c4ac73b1c63e in term 1.
I20260811 07:42:08.659080 3767 raft_consensus.cc:2454] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb8ee4a46d8e4c81b946c4ac73b1c63e in term 1.
I20260811 07:42:08.659376 3844 leader_election.cc:302] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13, bb8ee4a46d8e4c81b946c4ac73b1c63e; no voters:
I20260811 07:42:08.659667 4117 raft_consensus.cc:2781] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:08.659898 4117 raft_consensus.cc:686] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 LEADER]: Becoming Leader. State: Replica: bb8ee4a46d8e4c81b946c4ac73b1c63e, State: Running, Role: LEADER
I20260811 07:42:08.660241 4117 consensus_queue.cc:237] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:08.660519 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.656236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:46585 (local address 127.18.72.194:41721)
0811 07:42:08.656347 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.656351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.656428 (+ 77us) server_negotiation.cc:407] Connection header received
0811 07:42:08.656584 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.656587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.656630 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.656702 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:08.657160 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.657875 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.659332 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.659881 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.660004 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.660049 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.660348 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.660416 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:42:08.660529 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.656193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41721 (local address 127.18.72.195:46585)
0811 07:42:08.656368 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.656382 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:08.656465 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:08.656945 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:08.656948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:08.656963 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:08.657041 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.657045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.658001 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.658004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.659179 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.659190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.660105 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.660109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.660216 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.660238 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:42:08.660328 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 07:42:08.660409 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 07:42:08.660986 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.656872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:34635 (local address 127.18.72.193:37751)
0811 07:42:08.657011 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.657017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.657033 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:42:08.657074 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.657076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.657120 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.657200 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:08.657486 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.658196 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.659724 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.660352 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.660448 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.660490 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.660819 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.660887 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:42:08.661000 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.656317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37751 (local address 127.18.72.195:34635)
0811 07:42:08.656641 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.656655 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:08.656728 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:08.657239 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:08.657242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:08.657260 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:08.657363 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.657369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.658469 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.658472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.659598 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:08.659607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.660577 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.660581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:08.660684 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.660722 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 07:42:08.660803 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 07:42:08.660868 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":115,"threads_started":1}
I20260811 07:42:08.661098 3910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2282181bd54d7e9175129504690b99" candidate_uuid: "51a2e2c0c77f462a933b0e2da82dec19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" is_pre_election: true
I20260811 07:42:08.662479 3767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2282181bd54d7e9175129504690b99" candidate_uuid: "51a2e2c0c77f462a933b0e2da82dec19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" is_pre_election: true
I20260811 07:42:08.662673 3767 raft_consensus.cc:2379] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51a2e2c0c77f462a933b0e2da82dec19 in current term 1: Already voted for candidate bb8ee4a46d8e4c81b946c4ac73b1c63e in this term.
I20260811 07:42:08.663232 3983 leader_election.cc:302] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51a2e2c0c77f462a933b0e2da82dec19; no voters: 2f2a890e7aee48dd94dc5fd89ae72d13, bb8ee4a46d8e4c81b946c4ac73b1c63e
I20260811 07:42:08.663667 4118 raft_consensus.cc:2726] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 07:42:08.664435 3614 catalog_manager.cc:5495] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e reported cstate change: term changed from 0 to 1, leader changed from <none> to bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194). New cstate: current_term: 1 leader_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } health_report { overall_health: UNKNOWN } } }
I20260811 07:42:08.762822 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.759351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40008 (local address 127.18.72.194:41721)
0811 07:42:08.759482 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:08.759486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:08.759500 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:42:08.759580 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:08.759582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:08.759637 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:08.759731 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:42:08.760099 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.760696 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.761497 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:08.761703 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:08.761778 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:08.762038 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:42:08.762459 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:42:08.762549 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:08.762609 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 07:42:08.762652 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:42:08.768901 4052 raft_consensus.cc:1261] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 FOLLOWER]: Refusing update from remote peer bb8ee4a46d8e4c81b946c4ac73b1c63e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:08.768909 3767 raft_consensus.cc:1261] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Refusing update from remote peer bb8ee4a46d8e4c81b946c4ac73b1c63e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:08.769526 4117 consensus_queue.cc:1035] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:08.769749 4123 consensus_queue.cc:1035] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:08.775090 4128 mvcc.cc:205] Tried to move back new op lower bound from 7317234191430094848 to 7317234190997311488. Current Snapshot: MvccSnapshot[applied={T|T < 7317234191430094848}]
I20260811 07:42:08.777556 4127 mvcc.cc:205] Tried to move back new op lower bound from 7317234191430094848 to 7317234190997311488. Current Snapshot: MvccSnapshot[applied={T|T < 7317234191430094848}]
I20260811 07:42:08.778383 4129 mvcc.cc:205] Tried to move back new op lower bound from 7317234191430094848 to 7317234190997311488. Current Snapshot: MvccSnapshot[applied={T|T < 7317234191430094848}]
I20260811 07:42:08.785789 3615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:08.785969 3615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:08.788573 3615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Sending DeleteTablet for 3 replicas of tablet 0a2282181bd54d7e9175129504690b99
I20260811 07:42:08.789301 3745 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a2282181bd54d7e9175129504690b99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:08 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:38370
I20260811 07:42:08.789345 4031 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a2282181bd54d7e9175129504690b99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:08 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:45260
I20260811 07:42:08.789350 3889 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a2282181bd54d7e9175129504690b99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:08 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:39994
I20260811 07:42:08.789891 4137 tablet_replica.cc:330] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: stopping tablet replica
I20260811 07:42:08.789898 4139 tablet_replica.cc:330] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: stopping tablet replica
I20260811 07:42:08.790071 4137 raft_consensus.cc:2229] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:08.790151 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 3655
I20260811 07:42:08.790243 4137 raft_consensus.cc:2258] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:08.790241 4139 raft_consensus.cc:2229] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:08.790556 4139 raft_consensus.cc:2258] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:08.793233 4139 ts_tablet_manager.cc:1905] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:08.795972 4139 ts_tablet_manager.cc:1918] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:08.796132 4138 tablet_replica.cc:330] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: stopping tablet replica
I20260811 07:42:08.796154 4139 log.cc:1196] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/0a2282181bd54d7e9175129504690b99
I20260811 07:42:08.796511 4139 ts_tablet_manager.cc:1939] T 0a2282181bd54d7e9175129504690b99 P 51a2e2c0c77f462a933b0e2da82dec19: Deleting consensus metadata
I20260811 07:42:08.796708 4138 raft_consensus.cc:2229] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:42:08.797221 4138 raft_consensus.cc:2258] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:08.798035 3598 catalog_manager.cc:4841] TS 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223): tablet 0a2282181bd54d7e9175129504690b99 (table test-table [id=69d609876c9d4ca998a606f5ce1f4e16]) successfully deleted
I20260811 07:42:08.798724 4138 ts_tablet_manager.cc:1905] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:08.800925 4138 ts_tablet_manager.cc:1918] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:08.801043 4138 log.cc:1196] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/0a2282181bd54d7e9175129504690b99
I20260811 07:42:08.801403 4138 ts_tablet_manager.cc:1939] T 0a2282181bd54d7e9175129504690b99 P bb8ee4a46d8e4c81b946c4ac73b1c63e: Deleting consensus metadata
I20260811 07:42:08.802537 3600 catalog_manager.cc:4841] TS bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721): tablet 0a2282181bd54d7e9175129504690b99 (table test-table [id=69d609876c9d4ca998a606f5ce1f4e16]) successfully deleted
W20260811 07:42:08.803148 3602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:37751 (error 108)
I20260811 07:42:08.804266 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:08.803861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37751 (local address 127.0.0.1:38386)
0811 07:42:08.804119 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:08.804179 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":70,"threads_started":1}
I20260811 07:42:08.804381 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 3820
W20260811 07:42:08.808473 3602 catalog_manager.cc:4584] TS 2f2a890e7aee48dd94dc5fd89ae72d13 (127.18.72.193:37751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a2282181bd54d7e9175129504690b99: Network error: Client connection negotiation failed: client connection to 127.18.72.193:37751: connect: Connection refused (error 111)
I20260811 07:42:08.814599 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 3963
I20260811 07:42:08.821604 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 3581
I20260811 07:42:28.829659 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:42967
--webserver_interface=127.18.72.254
--webserver_port=41963
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:28.940796 4143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:28.941082 4143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:28.941162 4143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:28.945633 4143 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 07:42:28.945705 4143 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 07:42:28.945732 4143 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 07:42:28.945766 4143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:28.945813 4143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:28.945873 4143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:28.945906 4143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:28.945940 4143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:28.950997 4143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:42967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=41963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:28.952034 4143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:28.953043 4143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:28.961730 4149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:28.961735 4143 server_base.cc:1034] running on GCE node
W20260811 07:42:28.961846 4148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:28.961673 4151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:28.962482 4143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:28.963516 4143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:28.964704 4143 hybrid_clock.cc:648] HybridClock initialized: now 1786434148964669 us; error 50 us; skew 500 ppm
Aug 11 07:42:28 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434148, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:28.967418 4143 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:42:28.968286 4143 webserver.cc:460] Webserver started at http://127.18.72.254:41963/ using document root <none> and password file <none>
I20260811 07:42:28.968827 4143 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:28.968926 4143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:28.972491 4143 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:28.974498 4158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:28.975608 4143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:42:28.975772 4143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "c14340c2c2f14e1c8ca60163e8233673"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "f2d12dfd9575765feada411b44030ade"
server_key_iv: "aaf98d7971dafc863c0e95df1106b5bc"
server_key_version: "encryptionkey@0"
I20260811 07:42:28.976366 4143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:28.988015 4143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:28.990597 4143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:28.990736 4143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:28.997496 4143 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:42967
I20260811 07:42:28.997761 4213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:42967 every 8 connection(s)
I20260811 07:42:28.998502 4143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:29.002774 4214 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:42:29.004456 4214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Bootstrap starting.
I20260811 07:42:29.006922 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 4143
I20260811 07:42:29.007405 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:37751
--local_ip_for_outbound_sockets=127.18.72.193
--tserver_master_addrs=127.18.72.254:42967
--webserver_port=35971
--webserver_interface=127.18.72.193
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:42:29.026582 4214 log.cc:824] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:29.041373 4214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:42:29.041837 4214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Bootstrap complete.
I20260811 07:42:29.047546 4214 raft_consensus.cc:348] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:42:29.048054 4214 raft_consensus.cc:729] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14340c2c2f14e1c8ca60163e8233673, State: Initialized, Role: FOLLOWER
I20260811 07:42:29.048624 4214 consensus_queue.cc:260] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:42:29.048780 4214 raft_consensus.cc:388] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:29.048859 4214 raft_consensus.cc:482] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:29.048954 4214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 2 FOLLOWER]: Advancing to term 3
I20260811 07:42:29.051415 4214 raft_consensus.cc:504] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:42:29.051824 4214 leader_election.cc:302] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c14340c2c2f14e1c8ca60163e8233673; no voters:
I20260811 07:42:29.052205 4214 leader_election.cc:288] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 07:42:29.052335 4218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 07:42:29.052611 4218 raft_consensus.cc:686] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [term 3 LEADER]: Becoming Leader. State: Replica: c14340c2c2f14e1c8ca60163e8233673, State: Running, Role: LEADER
I20260811 07:42:29.052969 4218 consensus_queue.cc:237] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } }
I20260811 07:42:29.053472 4214 sys_catalog.cc:564] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:29.054545 4220 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c14340c2c2f14e1c8ca60163e8233673. Latest consensus state: current_term: 3 leader_uuid: "c14340c2c2f14e1c8ca60163e8233673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } } }
I20260811 07:42:29.054592 4219 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c14340c2c2f14e1c8ca60163e8233673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14340c2c2f14e1c8ca60163e8233673" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 42967 } } }
I20260811 07:42:29.054658 4220 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:29.054658 4219 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:29.055053 4227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:29.060256 4227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:29.061188 4227 catalog_manager.cc:1259] Loaded cluster ID: 153495ce3e3f4612b5f9d21ce36d3593
I20260811 07:42:29.061259 4227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:29.063820 4227 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:29.064591 4227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Loaded TSK: 0
I20260811 07:42:29.064678 4227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c14340c2c2f14e1c8ca60163e8233673: Loaded TSK: 1
F20260811 07:42:29.065500 4227 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f10700a69ad google::LogMessage::Fail() at ??:0
@ 0x7f10700a988a google::LogMessage::SendToLog() at ??:0
@ 0x7f10700a64de google::LogMessage::Flush() at ??:0
@ 0x7f10700a7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1072d6a18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1072d62f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f107aa0b7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f107a9e599c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f107a9e6178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f107aa27df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f107aa4682b std::function<>::operator()() at ??:0
@ 0x7f107a9e5b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f107a9e6f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f107a9e3579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f107aa27640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f10722f83d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f10722f8c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f10722fa3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f10722e9512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f107392d6db start_thread at ??:0
@ 0x7f106eba461f clone at ??:0
W20260811 07:42:29.172837 4216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:29.173213 4216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:29.173305 4216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:29.179459 4216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:29.179546 4216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:29.179669 4216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:42:29.187573 4216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:37751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=35971
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:29.188905 4216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:29.190261 4216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:29.201469 4268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:29.201479 4265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:29.201752 4216 server_base.cc:1034] running on GCE node
W20260811 07:42:29.201495 4266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:29.202255 4216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:29.202842 4216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:29.204017 4216 hybrid_clock.cc:648] HybridClock initialized: now 1786434149204003 us; error 31 us; skew 500 ppm
Aug 11 07:42:29 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434149, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:29.206924 4216 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:42:29.207782 4216 webserver.cc:460] Webserver started at http://127.18.72.193:35971/ using document root <none> and password file <none>
I20260811 07:42:29.208365 4216 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:29.208428 4216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:29.212287 4216 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:42:29.214536 4276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:29.216197 4216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:42:29.216337 4216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "2f2a890e7aee48dd94dc5fd89ae72d13"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "0a0b9169ac0a2617c603030f3b574573"
server_key_iv: "9c909d9d7b0bf045609be936d823d0a8"
server_key_version: "encryptionkey@0"
I20260811 07:42:29.216975 4216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:29.232651 4216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:29.235782 4216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:29.236020 4216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:29.236599 4216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:29.237685 4285 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 07:42:29.243151 4216 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:42:29.243229 4216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 07:42:29.243335 4216 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:42:29.244982 4216 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:42:29.245052 4216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260811 07:42:29.245373 4285 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Bootstrap starting.
I20260811 07:42:29.255769 4216 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:37751
I20260811 07:42:29.256032 4396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:37751 every 8 connection(s)
I20260811 07:42:29.256798 4216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:42:29.258824 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 4216
I20260811 07:42:29.259341 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:41721
--local_ip_for_outbound_sockets=127.18.72.194
--tserver_master_addrs=127.18.72.254:42967
--webserver_port=34045
--webserver_interface=127.18.72.194
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:42:29.269236 4285 log.cc:824] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:29.276263 4285 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:42:29.276641 4285 tablet_bootstrap.cc:492] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Bootstrap complete.
I20260811 07:42:29.277834 4285 ts_tablet_manager.cc:1397] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Time spent bootstrapping tablet: real 0.033s user 0.028s sys 0.000s
I20260811 07:42:29.280236 4285 raft_consensus.cc:348] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:29.280668 4285 raft_consensus.cc:729] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2a890e7aee48dd94dc5fd89ae72d13, State: Initialized, Role: FOLLOWER
I20260811 07:42:29.281145 4285 consensus_queue.cc:260] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:29.282069 4285 ts_tablet_manager.cc:1428] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260811 07:42:29.332756 4399 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:42:29.258800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:41629)
0811 07:42:29.260346 (+ 1546us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.260405 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:29.261606 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:29.331749 (+ 70143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1283,"thread_start_us":154,"threads_started":1}
W20260811 07:42:29.333334 4397 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:42967 (0 consecutive failures): Network error: Failed to ping master at 127.18.72.254:42967: Client connection negotiation failed: client connection to 127.18.72.254:42967: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 07:42:29.334136 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:29.333849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:46393)
0811 07:42:29.333995 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.334071 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 07:42:29.334818 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:29.334602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:56793)
0811 07:42:29.334711 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.334761 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 07:42:29.334991 4397 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 07:42:29.393342 4400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:29.393574 4400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:29.393642 4400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:29.397539 4400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:29.397591 4400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:29.397665 4400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:42:29.402810 4400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:41721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=34045
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:29.403748 4400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:29.404773 4400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:29.413882 4411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:29.413888 4408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:29.413862 4409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:29.414080 4400 server_base.cc:1034] running on GCE node
I20260811 07:42:29.414431 4400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:29.415060 4400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:29.416235 4400 hybrid_clock.cc:648] HybridClock initialized: now 1786434149416210 us; error 36 us; skew 500 ppm
Aug 11 07:42:29 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434149, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:29.418890 4400 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:42:29.419752 4400 webserver.cc:460] Webserver started at http://127.18.72.194:34045/ using document root <none> and password file <none>
I20260811 07:42:29.420337 4400 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:29.420401 4400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:29.424036 4400 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 07:42:29.425904 4418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:29.427037 4400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:29.427125 4400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "608699df4344e063c118e22e626bfa31"
server_key_iv: "9128e87b0ecf92f718eb0007d17b34e5"
server_key_version: "encryptionkey@0"
I20260811 07:42:29.427584 4400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:29.445104 4400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:29.447606 4400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:29.447763 4400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:29.448333 4400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:29.449091 4400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:29.449142 4400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:29.449180 4400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:29.449203 4400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:29.457983 4400 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:41721
I20260811 07:42:29.458016 4537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:41721 every 8 connection(s)
I20260811 07:42:29.459278 4400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:42:29.463862 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:29.461670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:52609)
0811 07:42:29.462207 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.462362 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":105,"threads_started":1}
W20260811 07:42:29.464754 4538 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:42967 (0 consecutive failures): Network error: Failed to ping master at 127.18.72.254:42967: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
I20260811 07:42:29.465708 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:29.465400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:37211)
0811 07:42:29.465577 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.465634 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 07:42:29.466452 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:29.466250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:47477)
0811 07:42:29.466347 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.466394 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 07:42:29.466624 4538 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:42:29.469002 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 4400
I20260811 07:42:29.469444 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:42223
--local_ip_for_outbound_sockets=127.18.72.195
--tserver_master_addrs=127.18.72.254:42967
--webserver_port=33201
--webserver_interface=127.18.72.195
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:46881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:29.582386 4541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:29.582633 4541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:29.582674 4541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:29.586601 4541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:29.586653 4541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:29.586753 4541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:42:29.591802 4541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:46881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:42223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=33201
--tserver_master_addrs=127.18.72.254:42967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:29.592793 4541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:29.593809 4541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:29.603392 4548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:29.603410 4550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:29.603519 4541 server_base.cc:1034] running on GCE node
W20260811 07:42:29.603407 4547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:29.604120 4541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:29.604734 4541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:29.605937 4541 hybrid_clock.cc:648] HybridClock initialized: now 1786434149605905 us; error 41 us; skew 500 ppm
Aug 11 07:42:29 dist-test-slave-zkpd krb5kdc[3007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434149, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:29.608675 4541 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:42:29.609519 4541 webserver.cc:460] Webserver started at http://127.18.72.195:33201/ using document root <none> and password file <none>
I20260811 07:42:29.610080 4541 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:29.610172 4541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:29.613907 4541 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:42:29.615849 4557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:29.617060 4541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:29.617190 4541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "51a2e2c0c77f462a933b0e2da82dec19"
format_stamp: "Formatted at 2026-08-11 07:41:46 on dist-test-slave-zkpd"
server_key: "caa04c2651166c58caa857e5d8f9d7c9"
server_key_iv: "f43526a9e1a3e70f9f118debd95a96ca"
server_key_version: "encryptionkey@0"
I20260811 07:42:29.617746 4541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:29.647287 4541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:29.650231 4541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:29.650435 4541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:29.651048 4541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:29.651877 4541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:29.651969 4541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:29.652047 4541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:29.652091 4541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:29.661293 4541 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:42223
I20260811 07:42:29.661324 4684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:42223 every 8 connection(s)
I20260811 07:42:29.662313 4541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:42:29.666229 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:29.664517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:57959)
0811 07:42:29.664982 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.665083 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
W20260811 07:42:29.666751 4685 heartbeater.cc:644] Failed to heartbeat to 127.18.72.254:42967 (0 consecutive failures): Network error: Failed to ping master at 127.18.72.254:42967: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
I20260811 07:42:29.667317 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:29.667121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:57071)
0811 07:42:29.667222 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.667266 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 07:42:29.667665 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 4541
I20260811 07:42:29.668260 4688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:29.667785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:53153)
0811 07:42:29.668078 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:29.668139 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":112,"threads_started":1}
W20260811 07:42:29.668465 4685 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:42:30.335954 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:30.335493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:52081)
0811 07:42:30.335745 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:30.335798 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260811 07:42:30.467679 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:30.467220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:35845)
0811 07:42:30.467534 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:30.467585 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":63,"threads_started":1}
I20260811 07:42:30.669297 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:30.668938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:37093)
0811 07:42:30.669154 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:30.669204 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":66,"threads_started":1}
I20260811 07:42:30.861631 4693 raft_consensus.cc:482] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:30.861891 4693 raft_consensus.cc:504] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:30.863061 4693 leader_election.cc:288] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223), bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
Aug 11 07:42:30 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434149, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.195@KRBTEST.COM
Aug 11 07:42:30 dist-test-slave-zkpd krb5kdc[3007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434149, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.194@KRBTEST.COM
I20260811 07:42:30.877655 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:30.863299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41721 (local address 127.18.72.193:43959)
0811 07:42:30.863643 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:30.863659 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:30.863765 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:30.865688 (+ 1923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:30.865691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:30.865941 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:30.866328 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:30.866343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:30.867824 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:30.867827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:30.868811 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:30.868820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:30.869570 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:30.869574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:30.869820 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:30.870466 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:30.870482 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:30.872516 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:30.874758 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:30.874761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:30.874764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:30.875110 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:30.875497 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:30.875500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:30.875503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:30.875722 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:30.876474 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:30.876481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:30.876879 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 07:42:30.877099 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 07:42:30.877453 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":185,"spinlock_wait_cycles":33664,"thread_start_us":70,"threads_started":1}
I20260811 07:42:30.877737 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:30.863524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:42827 (local address 127.18.72.195:42223)
0811 07:42:30.863877 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:30.863916 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:30.863970 (+ 54us) server_negotiation.cc:407] Connection header received
0811 07:42:30.864666 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:30.864672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:30.865040 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:30.865316 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:30.866547 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:30.867669 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:30.868941 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:30.869433 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:30.869615 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:30.872698 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:30.872702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:30.872713 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:30.872740 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:30.874503 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:30.875233 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:30.875239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:30.875245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:30.875405 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:30.875826 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:30.875829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:30.875831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:30.876357 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:30.876487 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:30.877236 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 07:42:30.877542 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260811 07:42:30.877980 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:30.863207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:42223 (local address 127.18.72.193:42827)
0811 07:42:30.863441 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:30.863456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:30.863581 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:30.865422 (+ 1841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:30.865429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:30.865911 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:30.866357 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:30.866369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:30.867822 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:30.867825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:30.868798 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:30.868808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:30.869560 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:30.869564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:30.869819 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:30.870517 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:30.870529 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:30.872153 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:30.874631 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:30.874637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:30.874651 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:30.875110 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:30.875497 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:30.875500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:30.875503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:30.875722 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:30.876474 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:30.876481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:30.876879 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 07:42:30.877098 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 07:42:30.877772 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":490,"spinlock_wait_cycles":39936,"thread_start_us":73,"threads_started":1}
I20260811 07:42:30.878444 4697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:30.863637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:43959 (local address 127.18.72.194:41721)
0811 07:42:30.864059 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:30.864080 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:30.864113 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:42:30.864748 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:30.864754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:30.865046 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:30.865343 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:30.866510 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:30.867666 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:30.868936 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:30.869433 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:30.869609 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:30.872688 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:30.872693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:30.872705 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:30.872731 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:30.874660 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:30.875233 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:30.875239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:30.875246 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:30.875392 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:30.875826 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:30.875829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:30.875831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:30.876346 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:30.876487 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:30.877428 (+ 941us) server_negotiation.cc:299] Negotiation successful
0811 07:42:30.878291 (+ 863us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":101,"threads_started":1}
W20260811 07:42:30.880390 4277 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
W20260811 07:42:30.880570 4278 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
I20260811 07:42:30.880650 4278 leader_election.cc:302] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13; no voters: 51a2e2c0c77f462a933b0e2da82dec19, bb8ee4a46d8e4c81b946c4ac73b1c63e
I20260811 07:42:30.880877 4693 raft_consensus.cc:2726] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:42:31.336939 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:31.336650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:34553)
0811 07:42:31.336790 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:31.336853 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 07:42:31.468708 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:31.468330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:51231)
0811 07:42:31.468574 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:31.468626 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
I20260811 07:42:31.670399 4699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:31.669947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:33743)
0811 07:42:31.670250 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:31.670314 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":82,"threads_started":1}
I20260811 07:42:32.338059 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:32.337634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:36949)
0811 07:42:32.337888 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:32.337961 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":71,"threads_started":1}
I20260811 07:42:32.469861 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:32.469437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:43407)
0811 07:42:32.469697 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:32.469752 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":76,"threads_started":1}
I20260811 07:42:32.539283 4702 raft_consensus.cc:482] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:32.539462 4702 raft_consensus.cc:504] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:32.540002 4702 leader_election.cc:288] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223), bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
W20260811 07:42:32.540817 4278 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
W20260811 07:42:32.540934 4277 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
I20260811 07:42:32.541051 4277 leader_election.cc:302] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13; no voters: 51a2e2c0c77f462a933b0e2da82dec19, bb8ee4a46d8e4c81b946c4ac73b1c63e
I20260811 07:42:32.541234 4702 raft_consensus.cc:2726] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:42:32.671562 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:32.671162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:46871)
0811 07:42:32.671423 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:32.671476 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260811 07:42:33.339246 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:33.338790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:53553)
0811 07:42:33.339081 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:33.339149 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":100,"threads_started":1}
I20260811 07:42:33.470970 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:33.470608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:33605)
0811 07:42:33.470835 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:33.470887 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260811 07:42:33.672622 4706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:33.672244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:41015)
0811 07:42:33.672485 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:33.672538 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":86,"threads_started":1}
I20260811 07:42:34.340592 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:34.340151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:46065)
0811 07:42:34.340448 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:34.340501 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":71,"threads_started":1}
I20260811 07:42:34.472244 4708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:34.471826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:41407)
0811 07:42:34.472102 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:34.472153 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":94,"threads_started":1}
I20260811 07:42:34.673875 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:34.673437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:39649)
0811 07:42:34.673729 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:34.673788 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":71,"threads_started":1}
I20260811 07:42:35.049183 4710 raft_consensus.cc:482] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:35.049321 4710 raft_consensus.cc:504] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:35.049808 4710 leader_election.cc:288] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223), bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
W20260811 07:42:35.050694 4278 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
W20260811 07:42:35.050796 4277 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
I20260811 07:42:35.050853 4278 leader_election.cc:302] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13; no voters: 51a2e2c0c77f462a933b0e2da82dec19, bb8ee4a46d8e4c81b946c4ac73b1c63e
I20260811 07:42:35.051030 4710 raft_consensus.cc:2726] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:42:35.341825 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:35.341357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:52461)
0811 07:42:35.341665 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:35.341729 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":97,"threads_started":1}
I20260811 07:42:35.473272 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:35.472904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:51055)
0811 07:42:35.473134 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:35.473189 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260811 07:42:35.675055 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:35.674601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:42813)
0811 07:42:35.674908 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:35.674966 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":89,"threads_started":1}
I20260811 07:42:36.343171 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:36.342713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:54441)
0811 07:42:36.342978 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:36.343033 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":72,"threads_started":1}
I20260811 07:42:36.474409 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:36.473987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:42121)
0811 07:42:36.474256 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:36.474309 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":67,"threads_started":1}
I20260811 07:42:36.676306 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:36.675799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:47465)
0811 07:42:36.676146 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:36.676198 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":136,"threads_started":1}
I20260811 07:42:37.344413 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:37.343987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:47749)
0811 07:42:37.344259 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:37.344315 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":92,"threads_started":1}
I20260811 07:42:37.475560 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:37.475154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:39571)
0811 07:42:37.475417 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:37.475471 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":92,"threads_started":1}
I20260811 07:42:37.677373 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:37.676979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:55079)
0811 07:42:37.677216 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:37.677268 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260811 07:42:37.680151 4720 raft_consensus.cc:482] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:37.680297 4720 raft_consensus.cc:504] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:37.680742 4720 leader_election.cc:288] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223), bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
W20260811 07:42:37.681425 4277 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
W20260811 07:42:37.681577 4278 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
I20260811 07:42:37.681648 4278 leader_election.cc:302] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13; no voters: 51a2e2c0c77f462a933b0e2da82dec19, bb8ee4a46d8e4c81b946c4ac73b1c63e
I20260811 07:42:37.681761 4720 raft_consensus.cc:2726] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:42:38.345572 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:38.345147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:54677)
0811 07:42:38.345421 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:38.345477 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
I20260811 07:42:38.476672 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:38.476287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:36023)
0811 07:42:38.476532 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:38.476582 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
I20260811 07:42:38.678547 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:38.678094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:42529)
0811 07:42:38.678396 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:38.678454 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":79,"threads_started":1}
I20260811 07:42:39.346897 4724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:39.346445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:48569)
0811 07:42:39.346735 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:39.346789 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
I20260811 07:42:39.477806 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:39.477405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:36361)
0811 07:42:39.477659 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:39.477713 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260811 07:42:39.679663 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:39.679286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:48821)
0811 07:42:39.679518 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:39.679570 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
I20260811 07:42:40.348071 4727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:40.347603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:46105)
0811 07:42:40.347870 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:40.347975 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":73,"threads_started":1}
I20260811 07:42:40.478996 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:40.478555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:52503)
0811 07:42:40.478836 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:40.478900 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":69,"threads_started":1}
I20260811 07:42:40.680850 4729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:40.680419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:52929)
0811 07:42:40.680683 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:40.680739 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":73,"threads_started":1}
I20260811 07:42:41.349301 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:41.348837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:39389)
0811 07:42:41.349140 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:41.349200 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":87,"threads_started":1}
I20260811 07:42:41.480268 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:41.479817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:49769)
0811 07:42:41.480126 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:41.480178 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":145,"threads_started":1}
I20260811 07:42:41.682065 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:41.681684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:50157)
0811 07:42:41.681925 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:41.681976 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":88,"threads_started":1}
I20260811 07:42:42.350638 4733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:42.350189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:47489)
0811 07:42:42.350481 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:42.350542 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":81,"threads_started":1}
I20260811 07:42:42.481407 4734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:42.481028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:54771)
0811 07:42:42.481263 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:42.481317 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":74,"threads_started":1}
I20260811 07:42:42.683238 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:42.682834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:48659)
0811 07:42:42.683097 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:42.683150 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":88,"threads_started":1}
I20260811 07:42:42.738034 4736 raft_consensus.cc:482] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:42.738168 4736 raft_consensus.cc:504] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51a2e2c0c77f462a933b0e2da82dec19" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 42223 } } peers { permanent_uuid: "bb8ee4a46d8e4c81b946c4ac73b1c63e" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41721 } } peers { permanent_uuid: "2f2a890e7aee48dd94dc5fd89ae72d13" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37751 } }
I20260811 07:42:42.738658 4736 leader_election.cc:288] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223), bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721)
W20260811 07:42:42.739367 4277 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 51a2e2c0c77f462a933b0e2da82dec19 (127.18.72.195:42223): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
W20260811 07:42:42.739488 4278 leader_election.cc:341] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb8ee4a46d8e4c81b946c4ac73b1c63e (127.18.72.194:41721): Not found: Tablet not found: 0a2282181bd54d7e9175129504690b99
I20260811 07:42:42.739614 4278 leader_election.cc:302] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f2a890e7aee48dd94dc5fd89ae72d13; no voters: 51a2e2c0c77f462a933b0e2da82dec19, bb8ee4a46d8e4c81b946c4ac73b1c63e
I20260811 07:42:42.739764 4736 raft_consensus.cc:2726] T 0a2282181bd54d7e9175129504690b99 P 2f2a890e7aee48dd94dc5fd89ae72d13 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:42:43.351879 4737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:43.351429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:60083)
0811 07:42:43.351728 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:43.351782 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":82,"threads_started":1}
I20260811 07:42:43.482473 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:43.482086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:55085)
0811 07:42:43.482329 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:43.482382 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":74,"threads_started":1}
I20260811 07:42:43.684478 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:43.684035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:52195)
0811 07:42:43.684325 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:43.684384 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":108,"threads_started":1}
I20260811 07:42:44.353307 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:44.352902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.193:43313)
0811 07:42:44.353159 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:44.353212 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260811 07:42:44.483687 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:44.483308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.194:56961)
0811 07:42:44.483546 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:44.483595 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
I20260811 07:42:44.675356 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 4216
I20260811 07:42:44.685197 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 4400
I20260811 07:42:44.687414 4742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:44.685388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:42967 (local address 127.18.72.195:33713)
0811 07:42:44.687179 (+ 1791us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:44.687248 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.254:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1682,"thread_start_us":73,"threads_started":1}
I20260811 07:42:44.692152 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 4541
2026-08-11T07:42:44Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59896 ms)
[----------] 30 tests from SecurityITest (243725 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:42:44 dist-test-slave-zkpd krb5kdc[4745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:42:44 dist-test-slave-zkpd krb5kdc[4745](info): set up 2 sockets
Aug 11 07:42:44 dist-test-slave-zkpd krb5kdc[4745](info): commencing operation
krb5kdc: starting...
W20260811 07:42:45.769033 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:42:45 dist-test-slave-zkpd krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:42:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:42:45.893085 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:37687
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:37687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:46.004500 4760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:46.004783 4760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:46.004854 4760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:46.008788 4760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:46.008859 4760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:46.008898 4760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:46.008955 4760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:46.008993 4760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:46.014112 4760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:37687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:37687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:46.015072 4760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:46.016083 4760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:42:46.024210 4760 server_base.cc:1034] running on GCE node
W20260811 07:42:46.024152 4769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:46.024163 4766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:46.024171 4767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:46.025060 4760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:46.026090 4760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:46.027275 4760 hybrid_clock.cc:648] HybridClock initialized: now 1786434166027260 us; error 40 us; skew 500 ppm
Aug 11 07:42:46 dist-test-slave-zkpd krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:46.029992 4760 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:42:46.030769 4760 webserver.cc:460] Webserver started at http://127.18.72.254:39151/ using document root <none> and password file <none>
I20260811 07:42:46.031273 4760 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:46.031332 4760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:46.031488 4760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:46.033551 4760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "bd12c9c685c2405589e061da9a082659"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "876ddad8f439a5003d824ccb22143dc9"
server_key_iv: "767cfd4f068856bdee12746a03ec7a92"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.034083 4760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "bd12c9c685c2405589e061da9a082659"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "876ddad8f439a5003d824ccb22143dc9"
server_key_iv: "767cfd4f068856bdee12746a03ec7a92"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.037748 4760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 07:42:46.040491 4776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.041846 4760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:42:46.041954 4760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "bd12c9c685c2405589e061da9a082659"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "876ddad8f439a5003d824ccb22143dc9"
server_key_iv: "767cfd4f068856bdee12746a03ec7a92"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.042098 4760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:46.056823 4760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:46.059492 4760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:46.059692 4760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:46.066761 4760 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:37687
I20260811 07:42:46.066766 4833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:37687 every 8 connection(s)
I20260811 07:42:46.067662 4760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:46.068730 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 4760
I20260811 07:42:46.068892 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:42:46.069180 18723 external_mini_cluster.cc:1428] Setting key ad47f0f2de138f2a17a866e1083e17e3
I20260811 07:42:46.070869 4834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:46.077618 4834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659: Bootstrap starting.
Aug 11 07:42:46 dist-test-slave-zkpd krb5kdc[4745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:46.080116 4834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:46.081053 4834 log.cc:824] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:46.083369 4834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659: No bootstrap required, opened a new log
I20260811 07:42:46.084570 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.070584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36380 (local address 127.18.72.254:37687)
0811 07:42:46.071021 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:46.071041 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:46.071081 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:42:46.071946 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:46.071971 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:46.072378 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:46.072731 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:46.073403 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.074919 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.075822 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.076544 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.076715 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.079418 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:46.079425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:46.079440 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:46.079471 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:46.081521 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.082038 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.082043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.082049 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.082176 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.082437 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.082440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.082442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.082931 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:46.083102 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:46.083332 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 07:42:46.083626 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260811 07:42:46.087167 4834 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd12c9c685c2405589e061da9a082659" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37687 } }
I20260811 07:42:46.087410 4834 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:46.087457 4834 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd12c9c685c2405589e061da9a082659, State: Initialized, Role: FOLLOWER
I20260811 07:42:46.087853 4834 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd12c9c685c2405589e061da9a082659" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37687 } }
I20260811 07:42:46.088025 4834 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:46.088075 4834 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:46.088145 4834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:46.088927 4834 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd12c9c685c2405589e061da9a082659" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37687 } }
I20260811 07:42:46.089219 4834 leader_election.cc:302] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd12c9c685c2405589e061da9a082659; no voters:
I20260811 07:42:46.089476 4834 leader_election.cc:288] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:42:46.089622 4839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:46.090070 4839 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [term 1 LEADER]: Becoming Leader. State: Replica: bd12c9c685c2405589e061da9a082659, State: Running, Role: LEADER
I20260811 07:42:46.090421 4839 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd12c9c685c2405589e061da9a082659" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37687 } }
I20260811 07:42:46.090704 4834 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:46.092027 4841 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd12c9c685c2405589e061da9a082659. Latest consensus state: current_term: 1 leader_uuid: "bd12c9c685c2405589e061da9a082659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd12c9c685c2405589e061da9a082659" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37687 } } }
I20260811 07:42:46.092085 4840 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd12c9c685c2405589e061da9a082659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd12c9c685c2405589e061da9a082659" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 37687 } } }
I20260811 07:42:46.092161 4841 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:46.092161 4840 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:46.092651 4851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:46.095744 4851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:46.100890 4851 catalog_manager.cc:1347] Generated new cluster ID: 3019edc370214c2087c5989c7f6f97ea
I20260811 07:42:46.100955 4851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:46.113974 4851 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:42:46.114919 4851 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:46.119017 4851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659: Generated new TSK 0
I20260811 07:42:46.119665 4851 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:42:46.148367 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37687
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:46.261232 4862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:46.261502 4862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:46.261569 4862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:46.265564 4862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:46.265635 4862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:46.265720 4862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:42:46.270785 4862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:46.271766 4862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:46.272780 4862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:46.282380 4869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:46.282404 4868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:46.282407 4871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:46.282631 4862 server_base.cc:1034] running on GCE node
I20260811 07:42:46.283039 4862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:46.283629 4862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:46.284814 4862 hybrid_clock.cc:648] HybridClock initialized: now 1786434166284775 us; error 48 us; skew 500 ppm
Aug 11 07:42:46 dist-test-slave-zkpd krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:46.287415 4862 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:42:46.288302 4862 webserver.cc:460] Webserver started at http://127.18.72.193:45107/ using document root <none> and password file <none>
I20260811 07:42:46.288870 4862 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:46.288966 4862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:46.289194 4862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:46.291306 4862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "87cbb727b796410f8aa73767c2c4b951"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "5d7f63050b9e100b0b972cc05c596406"
server_key_iv: "f2a6f3d78260fbf18ab5090fed855bba"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.291864 4862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "87cbb727b796410f8aa73767c2c4b951"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "5d7f63050b9e100b0b972cc05c596406"
server_key_iv: "f2a6f3d78260fbf18ab5090fed855bba"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.295526 4862 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:42:46.297871 4878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.299088 4862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:46.299219 4862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "87cbb727b796410f8aa73767c2c4b951"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "5d7f63050b9e100b0b972cc05c596406"
server_key_iv: "f2a6f3d78260fbf18ab5090fed855bba"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.299324 4862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:46.330726 4862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:46.334388 4862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:46.334582 4862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:46.335166 4862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:46.336019 4862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:46.336087 4862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.336151 4862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:46.336199 4862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.344817 4862 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:39283
I20260811 07:42:46.344853 5003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:39283 every 8 connection(s)
I20260811 07:42:46.345678 4862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
Aug 11 07:42:46 dist-test-slave-zkpd krb5kdc[4745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:46.355152 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 4862
I20260811 07:42:46.355288 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:42:46.355610 18723 external_mini_cluster.cc:1428] Setting key 7755492f21b43a2121bd06ea76734e2c
I20260811 07:42:46.359421 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.347449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:33609 (local address 127.18.72.254:37687)
0811 07:42:46.347578 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:46.347582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:46.348391 (+ 809us) server_negotiation.cc:407] Connection header received
0811 07:42:46.349718 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:46.349721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:46.349781 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:46.349901 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:46.350996 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.351515 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.352664 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.352885 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.352985 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.355468 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:46.355471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:46.355473 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:46.355496 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:46.357224 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.357801 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.357804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.357805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.357887 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.358260 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.358262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.358263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.358424 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:46.358497 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:46.359128 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 07:42:46.359228 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:42:46.360285 5006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.347797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37687 (local address 127.18.72.193:33609)
0811 07:42:46.348248 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:46.348289 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:46.349517 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:46.350070 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:46.350077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:46.350396 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:46.350859 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.350872 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.351638 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.351642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.352534 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.352544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.353021 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.353025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.353181 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.353760 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:46.353776 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:46.355305 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:46.357337 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:46.357343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:46.357357 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:46.357703 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:46.357973 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:46.357978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:46.357983 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:46.358170 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:46.358521 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:46.358529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:46.358842 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 07:42:46.359070 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 07:42:46.359423 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":113,"threads_started":1}
I20260811 07:42:46.361553 5004 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37687
I20260811 07:42:46.361780 5004 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:46.362247 5004 heartbeater.cc:511] Master 127.18.72.254:37687 requested a full tablet report, sending...
I20260811 07:42:46.364037 4793 ts_manager.cc:194] Registered new tserver with Master: 87cbb727b796410f8aa73767c2c4b951 (127.18.72.193:39283)
I20260811 07:42:46.365727 4793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:33609
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:42:46.384913 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37687
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:46.497929 5010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:46.498164 5010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:46.498205 5010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:46.502128 5010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:46.502197 5010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:46.502274 5010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:42:46.507402 5010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:46.508436 5010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:46.509400 5010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:46.518715 5017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:46.518777 5010 server_base.cc:1034] running on GCE node
W20260811 07:42:46.518724 5020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:46.518750 5016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:46.519296 5010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:46.519862 5010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:46.521075 5010 hybrid_clock.cc:648] HybridClock initialized: now 1786434166521049 us; error 42 us; skew 500 ppm
Aug 11 07:42:46 dist-test-slave-zkpd krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:46.523731 5010 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:42:46.524552 5010 webserver.cc:460] Webserver started at http://127.18.72.194:39785/ using document root <none> and password file <none>
I20260811 07:42:46.525095 5010 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:46.525156 5010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:46.525377 5010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:46.527283 5010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "b089cb3fa1924d54bd546cda326b925a"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "8caaca707dc03ec130e26cba749acf73"
server_key_iv: "61b5c5e570437c29c2cd1857ed9b5aa7"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.527822 5010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "b089cb3fa1924d54bd546cda326b925a"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "8caaca707dc03ec130e26cba749acf73"
server_key_iv: "61b5c5e570437c29c2cd1857ed9b5aa7"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.531395 5010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:42:46.533749 5028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.534947 5010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:46.535072 5010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "b089cb3fa1924d54bd546cda326b925a"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "8caaca707dc03ec130e26cba749acf73"
server_key_iv: "61b5c5e570437c29c2cd1857ed9b5aa7"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.535173 5010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:46.554612 5010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:46.557797 5010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:46.557981 5010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:46.558519 5010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:46.559319 5010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:46.559384 5010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.559453 5010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:46.559491 5010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.568521 5010 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:38231
I20260811 07:42:46.568563 5153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:38231 every 8 connection(s)
I20260811 07:42:46.569334 5010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:42:46.571724 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 5010
I20260811 07:42:46.571851 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:42:46.572202 18723 external_mini_cluster.cc:1428] Setting key a680e05a57ea14eb1ac846905eb0e559
Aug 11 07:42:46 dist-test-slave-zkpd krb5kdc[4745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:42:46.583478 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.570989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:42385 (local address 127.18.72.254:37687)
0811 07:42:46.571128 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:46.571132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:46.572022 (+ 890us) server_negotiation.cc:407] Connection header received
0811 07:42:46.573527 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:46.573530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:46.573582 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:46.573673 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:46.575186 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.575740 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.576920 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.577253 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.577339 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.580072 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:46.580076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:46.580078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:46.580100 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:46.581464 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.581971 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.581974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.581975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.582050 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.582397 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.582400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.582401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.582583 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:46.582684 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:46.583158 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 07:42:46.583277 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:42:46.584367 5156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.571346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37687 (local address 127.18.72.194:42385)
0811 07:42:46.571834 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:46.571895 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:46.573273 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:46.573869 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:46.573879 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:46.574374 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:46.575041 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.575063 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.575919 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.575927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.576799 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.576809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.577383 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.577386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.577523 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.578065 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:46.578080 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:46.579871 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:46.581581 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:46.581587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:46.581601 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:46.581859 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:46.582139 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:46.582143 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:46.582145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:46.582284 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:46.582683 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:46.582689 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:46.582929 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 07:42:46.583100 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 07:42:46.583428 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":157,"threads_started":1}
I20260811 07:42:46.585638 5154 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37687
I20260811 07:42:46.585881 5154 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
I20260811 07:42:46.586486 5154 heartbeater.cc:511] Master 127.18.72.254:37687 requested a full tablet report, sending...
I20260811 07:42:46.588019 4793 ts_manager.cc:194] Registered new tserver with Master: b089cb3fa1924d54bd546cda326b925a (127.18.72.194:38231)
I20260811 07:42:46.588582 4793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:42385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:42:46.601082 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37687
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:46.713754 5159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:46.713981 5159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:46.714020 5159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:46.717955 5159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:46.718009 5159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:46.718063 5159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:42:46.723078 5159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:37687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:46.723986 5159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:46.724956 5159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:46.734107 5166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:46.734385 5165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:46.734493 5159 server_base.cc:1034] running on GCE node
W20260811 07:42:46.734395 5168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:46.734922 5159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:46.735503 5159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:46.736657 5159 hybrid_clock.cc:648] HybridClock initialized: now 1786434166736642 us; error 25 us; skew 500 ppm
Aug 11 07:42:46 dist-test-slave-zkpd krb5kdc[4745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:46.739353 5159 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:42:46.740163 5159 webserver.cc:460] Webserver started at http://127.18.72.195:40087/ using document root <none> and password file <none>
I20260811 07:42:46.740653 5159 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:46.740705 5159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:46.740865 5159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:46.742753 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "7eb2533ed2c84945892d04cae3f4e57e"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "581454591753e41b3c8b677d5d25405f"
server_key_iv: "3fdd6ebbc306a9fe34079d8c40993486"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.743259 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "7eb2533ed2c84945892d04cae3f4e57e"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "581454591753e41b3c8b677d5d25405f"
server_key_iv: "3fdd6ebbc306a9fe34079d8c40993486"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.746942 5159 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:42:46.749187 5176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.750324 5159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:42:46.750473 5159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "7eb2533ed2c84945892d04cae3f4e57e"
format_stamp: "Formatted at 2026-08-11 07:42:46 on dist-test-slave-zkpd"
server_key: "581454591753e41b3c8b677d5d25405f"
server_key_iv: "3fdd6ebbc306a9fe34079d8c40993486"
server_key_version: "encryptionkey@0"
I20260811 07:42:46.750572 5159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:46.766266 5159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:46.769328 5159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:46.769558 5159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:46.770097 5159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:46.770869 5159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:46.770933 5159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.771001 5159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:46.771041 5159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:46.779944 5159 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:46253
I20260811 07:42:46.779953 5299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:46253 every 8 connection(s)
I20260811 07:42:46.780762 5159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:42:46.787235 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 5159
I20260811 07:42:46.787374 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:42:46.787662 18723 external_mini_cluster.cc:1428] Setting key 723e7e733d79ce3116a14d57770f6a75
Aug 11 07:42:46 dist-test-slave-zkpd krb5kdc[4745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434166, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:46.793969 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.782494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:54897 (local address 127.18.72.254:37687)
0811 07:42:46.782625 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:46.782629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:46.783404 (+ 775us) server_negotiation.cc:407] Connection header received
0811 07:42:46.784741 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:46.784743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:46.784789 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:46.784866 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:46.785931 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.786445 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.787868 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.788247 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.788336 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.790885 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:46.790888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:46.790890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:46.790931 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:46.792261 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.792727 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.792730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.792731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.792804 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.793125 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.793127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.793128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.793284 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:46.793369 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:46.793768 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 07:42:46.793861 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:42:46.794703 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.782831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:37687 (local address 127.18.72.195:54897)
0811 07:42:46.783248 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:46.783297 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:46.784550 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:46.785023 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:46.785029 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:46.785360 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:46.785810 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.785823 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.786545 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.786548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.787752 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.787765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.788399 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.788403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.788585 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.789234 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:46.789251 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:46.790718 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:46.792370 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:46.792375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:46.792386 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:46.792645 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:46.792918 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:46.792921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:46.792922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:46.793048 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:46.793369 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:46.793373 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:46.793577 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 07:42:46.793733 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 07:42:46.793969 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":109,"threads_started":1}
I20260811 07:42:46.795747 5300 heartbeater.cc:347] Connected to a master server at 127.18.72.254:37687
I20260811 07:42:46.795991 5300 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:46.796411 5300 heartbeater.cc:511] Master 127.18.72.254:37687 requested a full tablet report, sending...
I20260811 07:42:46.797371 4793 ts_manager.cc:194] Registered new tserver with Master: 7eb2533ed2c84945892d04cae3f4e57e (127.18.72.195:46253)
I20260811 07:42:46.797930 4793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:54897
I20260811 07:42:46.801159 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:42:46.809427 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.803095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36382 (local address 127.18.72.254:37687)
0811 07:42:46.803195 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:46.803198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:46.803367 (+ 169us) server_negotiation.cc:407] Connection header received
0811 07:42:46.803471 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:46.803472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:46.803502 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:46.803559 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:46.804258 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.804724 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.805515 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.805708 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.805779 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.806934 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:46.806937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:46.806938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:46.806958 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:46.808152 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.808508 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.808510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.808511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.808579 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:46.808807 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:46.808809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:46.808810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:46.808989 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:46.809087 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:46.809247 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 07:42:46.809328 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:42:46.811978 4793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:42:46.815384 4793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:42:46.829157 5319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.823974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:38231 (local address 127.0.0.1:35696)
0811 07:42:46.824970 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:46.824982 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:46.825069 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:46.825509 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:46.825512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:46.825529 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:46.825615 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.825624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.826752 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.826755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.827750 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.827759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.828639 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.828646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.828754 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.828790 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 07:42:46.828899 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 07:42:46.829008 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":935,"thread_start_us":87,"threads_started":1}
I20260811 07:42:46.829191 5318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.824146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.18.72.194:38231)
0811 07:42:46.824615 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:46.824631 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:46.825084 (+ 453us) server_negotiation.cc:407] Connection header received
0811 07:42:46.825172 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:46.825180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:46.825377 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:46.825524 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:46.825713 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.826619 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.827866 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.828503 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.828612 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.828750 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:46.828960 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 07:42:46.829055 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":140,"threads_started":1}
I20260811 07:42:46.829479 5315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.823340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33794 (local address 127.18.72.193:39283)
0811 07:42:46.823678 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:46.823695 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:46.823928 (+ 233us) server_negotiation.cc:407] Connection header received
0811 07:42:46.824035 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:46.824044 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:46.824272 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:46.824434 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:46.824961 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.826105 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.827711 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.828595 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.828693 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.828838 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:46.829230 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 07:42:46.829348 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":131,"threads_started":1}
I20260811 07:42:46.829499 5314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.823220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39283 (local address 127.0.0.1:33794)
0811 07:42:46.823639 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:46.823689 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:46.823831 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:46.824640 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:46.824646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:46.824667 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:46.824791 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.824800 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.826250 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.826254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.827575 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.827586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.828903 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.828907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.828986 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.829005 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 07:42:46.829072 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 07:42:46.829124 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":85,"threads_started":1}
I20260811 07:42:46.830195 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.824295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:46253 (local address 127.0.0.1:47254)
0811 07:42:46.825193 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:46.825201 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:46.825288 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:46.825746 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:46.825748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:46.825763 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:46.825867 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.825873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.827215 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.827219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.828341 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:46.828352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.829815 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.829818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:46.829916 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.829939 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 07:42:46.830019 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 07:42:46.830084 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"thread_start_us":96,"threads_started":1}
I20260811 07:42:46.830305 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:46.824477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47254 (local address 127.18.72.195:46253)
0811 07:42:46.824830 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:46.824845 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:46.825292 (+ 447us) server_negotiation.cc:407] Connection header received
0811 07:42:46.825389 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:46.825393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:46.825573 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:46.825728 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:46.825980 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.827067 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.828481 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:46.829292 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:46.829402 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:46.829536 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:46.830070 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 07:42:46.830166 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":168,"threads_started":1}
I20260811 07:42:46.832258 4934 tablet_service.cc:1467] Processing CreateTablet for tablet 044113a5c79e4cfdbcf9fb493475b947 (DEFAULT_TABLE table=test-table [id=3921bffd7f9e4dde86764373ef4e7819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:46.832263 5082 tablet_service.cc:1467] Processing CreateTablet for tablet 044113a5c79e4cfdbcf9fb493475b947 (DEFAULT_TABLE table=test-table [id=3921bffd7f9e4dde86764373ef4e7819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:46.833292 4934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044113a5c79e4cfdbcf9fb493475b947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:46.833315 5082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044113a5c79e4cfdbcf9fb493475b947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:46.834606 5228 tablet_service.cc:1467] Processing CreateTablet for tablet 044113a5c79e4cfdbcf9fb493475b947 (DEFAULT_TABLE table=test-table [id=3921bffd7f9e4dde86764373ef4e7819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:46.835613 5228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044113a5c79e4cfdbcf9fb493475b947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:46.840529 5321 tablet_bootstrap.cc:492] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951: Bootstrap starting.
I20260811 07:42:46.841233 5322 tablet_bootstrap.cc:492] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Bootstrap starting.
I20260811 07:42:46.842145 5323 tablet_bootstrap.cc:492] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Bootstrap starting.
I20260811 07:42:46.842896 5321 tablet_bootstrap.cc:654] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:46.844051 5323 tablet_bootstrap.cc:654] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:46.844079 5322 tablet_bootstrap.cc:654] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:46.844285 5321 log.cc:824] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:46.845297 5323 log.cc:824] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:46.845348 5322 log.cc:824] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:46.846570 5321 tablet_bootstrap.cc:492] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951: No bootstrap required, opened a new log
I20260811 07:42:46.846885 5321 ts_tablet_manager.cc:1397] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 07:42:46.847323 5322 tablet_bootstrap.cc:492] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: No bootstrap required, opened a new log
I20260811 07:42:46.847328 5323 tablet_bootstrap.cc:492] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: No bootstrap required, opened a new log
I20260811 07:42:46.847498 5323 ts_tablet_manager.cc:1397] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260811 07:42:46.847640 5322 ts_tablet_manager.cc:1397] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 07:42:46.850412 5322 raft_consensus.cc:348] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:46.850631 5322 raft_consensus.cc:374] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:46.850708 5322 raft_consensus.cc:729] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b089cb3fa1924d54bd546cda326b925a, State: Initialized, Role: FOLLOWER
I20260811 07:42:46.850813 5321 raft_consensus.cc:348] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:46.851055 5321 raft_consensus.cc:374] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:46.851154 5321 raft_consensus.cc:729] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87cbb727b796410f8aa73767c2c4b951, State: Initialized, Role: FOLLOWER
I20260811 07:42:46.851176 5322 consensus_queue.cc:260] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:46.851241 5323 raft_consensus.cc:348] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:46.851579 5323 raft_consensus.cc:374] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:46.851557 5321 consensus_queue.cc:260] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:46.851666 5323 raft_consensus.cc:729] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eb2533ed2c84945892d04cae3f4e57e, State: Initialized, Role: FOLLOWER
I20260811 07:42:46.851833 5154 heartbeater.cc:503] Master 127.18.72.254:37687 was elected leader, sending a full tablet report...
I20260811 07:42:46.852234 5004 heartbeater.cc:503] Master 127.18.72.254:37687 was elected leader, sending a full tablet report...
I20260811 07:42:46.852190 5323 consensus_queue.cc:260] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:46.852573 5321 ts_tablet_manager.cc:1428] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 07:42:46.853153 5322 ts_tablet_manager.cc:1428] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:42:46.853197 5323 ts_tablet_manager.cc:1428] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:42:46.853724 5300 heartbeater.cc:503] Master 127.18.72.254:37687 was elected leader, sending a full tablet report...
W20260811 07:42:47.032147 5301 tablet.cc:2369] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:42:47.070427 5155 tablet.cc:2369] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:42:47.097052 5005 tablet.cc:2369] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:42:47.290493 5330 raft_consensus.cc:482] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:47.290735 5330 raft_consensus.cc:504] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:47.292111 5330 leader_election.cc:288] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87cbb727b796410f8aa73767c2c4b951 (127.18.72.193:39283), b089cb3fa1924d54bd546cda326b925a (127.18.72.194:38231)
I20260811 07:42:47.295830 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.292382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39283 (local address 127.18.72.195:53241)
0811 07:42:47.292620 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:47.292635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:47.292739 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:47.293061 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:47.293065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:47.293084 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:47.293212 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:47.293218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.293964 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.293970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:47.294746 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:47.294755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.295392 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.295395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:47.295504 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:47.295529 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 07:42:47.295616 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:42:47.295693 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"spinlock_wait_cycles":3840}
I20260811 07:42:47.295830 5315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.292449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:53241 (local address 127.18.72.193:39283)
0811 07:42:47.292580 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:47.292584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:47.292697 (+ 113us) server_negotiation.cc:407] Connection header received
0811 07:42:47.292879 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:47.292883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:47.292944 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:47.293033 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:47.293329 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.293854 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.294886 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.295273 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.295400 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:47.295456 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:47.295635 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 07:42:47.295705 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:42:47.296111 5331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.292673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:38231 (local address 127.18.72.195:43935)
0811 07:42:47.292979 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:47.292989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:47.293109 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:47.293368 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:47.293372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:47.293387 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:47.293523 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:47.293528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.294237 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.294242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:47.295028 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:47.295038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.295667 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.295671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:47.295770 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:47.295792 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:42:47.295883 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 07:42:47.295965 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260811 07:42:47.296104 5318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.292837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:43935 (local address 127.18.72.194:38231)
0811 07:42:47.292965 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:47.292971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:47.293075 (+ 104us) server_negotiation.cc:407] Connection header received
0811 07:42:47.293218 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:47.293226 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:47.293271 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:47.293345 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:47.293626 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.294129 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.295157 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.295550 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.295675 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:47.295723 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:47.295928 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 07:42:47.296002 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:42:47.296947 5104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044113a5c79e4cfdbcf9fb493475b947" candidate_uuid: "7eb2533ed2c84945892d04cae3f4e57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b089cb3fa1924d54bd546cda326b925a" is_pre_election: true
I20260811 07:42:47.296949 4956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044113a5c79e4cfdbcf9fb493475b947" candidate_uuid: "7eb2533ed2c84945892d04cae3f4e57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87cbb727b796410f8aa73767c2c4b951" is_pre_election: true
I20260811 07:42:47.297281 4956 raft_consensus.cc:2454] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eb2533ed2c84945892d04cae3f4e57e in term 0.
I20260811 07:42:47.297298 5104 raft_consensus.cc:2454] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eb2533ed2c84945892d04cae3f4e57e in term 0.
I20260811 07:42:47.297804 5181 leader_election.cc:302] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7eb2533ed2c84945892d04cae3f4e57e, b089cb3fa1924d54bd546cda326b925a; no voters:
I20260811 07:42:47.298045 5330 raft_consensus.cc:2781] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:42:47.298146 5330 raft_consensus.cc:482] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:42:47.298202 5330 raft_consensus.cc:3037] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:47.299271 5330 raft_consensus.cc:504] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:47.299615 5330 leader_election.cc:288] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [CANDIDATE]: Term 1 election: Requested vote from peers 87cbb727b796410f8aa73767c2c4b951 (127.18.72.193:39283), b089cb3fa1924d54bd546cda326b925a (127.18.72.194:38231)
I20260811 07:42:47.300014 4956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044113a5c79e4cfdbcf9fb493475b947" candidate_uuid: "7eb2533ed2c84945892d04cae3f4e57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87cbb727b796410f8aa73767c2c4b951"
I20260811 07:42:47.300055 5104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044113a5c79e4cfdbcf9fb493475b947" candidate_uuid: "7eb2533ed2c84945892d04cae3f4e57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b089cb3fa1924d54bd546cda326b925a"
I20260811 07:42:47.300160 4956 raft_consensus.cc:3037] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:47.300175 5104 raft_consensus.cc:3037] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:47.301204 4956 raft_consensus.cc:2454] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eb2533ed2c84945892d04cae3f4e57e in term 1.
I20260811 07:42:47.301261 5104 raft_consensus.cc:2454] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eb2533ed2c84945892d04cae3f4e57e in term 1.
I20260811 07:42:47.301538 5181 leader_election.cc:302] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7eb2533ed2c84945892d04cae3f4e57e, 87cbb727b796410f8aa73767c2c4b951; no voters:
I20260811 07:42:47.301698 5330 raft_consensus.cc:2781] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:47.301910 5330 raft_consensus.cc:686] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 1 LEADER]: Becoming Leader. State: Replica: 7eb2533ed2c84945892d04cae3f4e57e, State: Running, Role: LEADER
I20260811 07:42:47.302260 5330 consensus_queue.cc:237] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } }
I20260811 07:42:47.305629 4792 catalog_manager.cc:5495] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eb2533ed2c84945892d04cae3f4e57e (127.18.72.195). New cstate: current_term: 1 leader_uuid: "7eb2533ed2c84945892d04cae3f4e57e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eb2533ed2c84945892d04cae3f4e57e" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46253 } health_report { overall_health: HEALTHY } } }
I20260811 07:42:47.398391 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.394139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.18.72.195:46253)
0811 07:42:47.394427 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:47.394432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:47.394446 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:42:47.394615 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:47.394618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:47.394679 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:47.394800 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:42:47.395053 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.395573 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.396524 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.396755 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.396863 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:47.397248 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:42:47.397931 (+ 683us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:42:47.398073 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:47.398147 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 07:42:47.398223 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
I20260811 07:42:47.404805 5104 raft_consensus.cc:1261] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 1 FOLLOWER]: Refusing update from remote peer 7eb2533ed2c84945892d04cae3f4e57e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:47.404794 4956 raft_consensus.cc:1261] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 1 FOLLOWER]: Refusing update from remote peer 7eb2533ed2c84945892d04cae3f4e57e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:47.405540 5330 consensus_queue.cc:1035] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b089cb3fa1924d54bd546cda326b925a" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 38231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:47.405771 5332 consensus_queue.cc:1035] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "87cbb727b796410f8aa73767c2c4b951" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:47.412171 5338 mvcc.cc:205] Tried to move back new op lower bound from 7317234349681242112 to 7317234349273346048. Current Snapshot: MvccSnapshot[applied={T|T < 7317234349681242112}]
I20260811 07:42:47.414685 5342 mvcc.cc:205] Tried to move back new op lower bound from 7317234349681242112 to 7317234349273346048. Current Snapshot: MvccSnapshot[applied={T|T < 7317234349681242112}]
I20260811 07:42:47.417098 5341 mvcc.cc:205] Tried to move back new op lower bound from 7317234349681242112 to 7317234349273346048. Current Snapshot: MvccSnapshot[applied={T|T < 7317234349681242112 or (T in {7317234349681242112})}]
I20260811 07:42:47.420477 4793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:47.420648 4793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:47.423098 4793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd12c9c685c2405589e061da9a082659: Sending DeleteTablet for 3 replicas of tablet 044113a5c79e4cfdbcf9fb493475b947
I20260811 07:42:47.423838 4934 tablet_service.cc:1514] Processing DeleteTablet for tablet 044113a5c79e4cfdbcf9fb493475b947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:33794
I20260811 07:42:47.423923 5082 tablet_service.cc:1514] Processing DeleteTablet for tablet 044113a5c79e4cfdbcf9fb493475b947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:35696
I20260811 07:42:47.424329 5350 tablet_replica.cc:330] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951: stopping tablet replica
I20260811 07:42:47.424386 5228 tablet_service.cc:1514] Processing DeleteTablet for tablet 044113a5c79e4cfdbcf9fb493475b947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:47 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:47254
I20260811 07:42:47.424659 5350 raft_consensus.cc:2229] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:47.424856 5352 tablet_replica.cc:330] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: stopping tablet replica
I20260811 07:42:47.425047 5350 raft_consensus.cc:2258] T 044113a5c79e4cfdbcf9fb493475b947 P 87cbb727b796410f8aa73767c2c4b951 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:47.425189 5352 raft_consensus.cc:2229] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:42:47.425338 5351 tablet_replica.cc:330] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: stopping tablet replica
I20260811 07:42:47.425431 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 4862
I20260811 07:42:47.426082 5351 raft_consensus.cc:2229] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:47.426266 5352 raft_consensus.cc:2258] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:47.426683 5351 raft_consensus.cc:2258] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:47.428258 5351 ts_tablet_manager.cc:1905] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:47.428315 5352 ts_tablet_manager.cc:1905] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:47.431059 5351 ts_tablet_manager.cc:1918] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:47.431146 5351 log.cc:1196] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/044113a5c79e4cfdbcf9fb493475b947
I20260811 07:42:47.431454 5351 ts_tablet_manager.cc:1939] T 044113a5c79e4cfdbcf9fb493475b947 P b089cb3fa1924d54bd546cda326b925a: Deleting consensus metadata
I20260811 07:42:47.432368 5352 ts_tablet_manager.cc:1918] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:47.432482 5352 log.cc:1196] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/044113a5c79e4cfdbcf9fb493475b947
I20260811 07:42:47.432878 5352 ts_tablet_manager.cc:1939] T 044113a5c79e4cfdbcf9fb493475b947 P 7eb2533ed2c84945892d04cae3f4e57e: Deleting consensus metadata
I20260811 07:42:47.435209 4780 catalog_manager.cc:4841] TS 7eb2533ed2c84945892d04cae3f4e57e (127.18.72.195:46253): tablet 044113a5c79e4cfdbcf9fb493475b947 (table test-table [id=3921bffd7f9e4dde86764373ef4e7819]) successfully deleted
I20260811 07:42:47.435364 4780 catalog_manager.cc:4841] TS b089cb3fa1924d54bd546cda326b925a (127.18.72.194:38231): tablet 044113a5c79e4cfdbcf9fb493475b947 (table test-table [id=3921bffd7f9e4dde86764373ef4e7819]) successfully deleted
W20260811 07:42:47.443094 4780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:39283 (error 108)
I20260811 07:42:47.443259 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 5010
I20260811 07:42:47.446715 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.444618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39283 (local address 127.0.0.1:33808)
0811 07:42:47.446498 (+ 1880us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:47.446583 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1779,"thread_start_us":65,"threads_started":1}
W20260811 07:42:47.447012 4780 catalog_manager.cc:4584] TS 87cbb727b796410f8aa73767c2c4b951 (127.18.72.193:39283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 044113a5c79e4cfdbcf9fb493475b947: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39283: connect: Connection refused (error 111)
I20260811 07:42:47.450870 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 5159
I20260811 07:42:47.457794 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 4760
2026-08-11T07:42:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2770 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T07:42:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:47Z Disabled control of system clock
I20260811 07:42:47.497495 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45015
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:45015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:42:47.610078 5358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:47.610321 5358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:47.610355 5358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:47.614378 5358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:47.614429 5358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:47.614445 5358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:47.614461 5358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:47.614499 5358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:47.619143 5358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:45015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:47.620220 5358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:47.621569 5358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:47.629935 5367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:47.629966 5365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:47.630169 5358 server_base.cc:1034] running on GCE node
W20260811 07:42:47.629938 5364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:47.630887 5358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:47.632010 5358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:47.633185 5358 hybrid_clock.cc:648] HybridClock initialized: now 1786434167633167 us; error 33 us; skew 500 ppm
I20260811 07:42:47.634963 5358 webserver.cc:460] Webserver started at http://127.18.72.254:43021/ using document root <none> and password file <none>
I20260811 07:42:47.635532 5358 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:47.635623 5358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:47.635854 5358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:47.637972 5358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "73a630174de649ae89e118ec7649a0e1"
format_stamp: "Formatted at 2026-08-11 07:42:47 on dist-test-slave-zkpd"
server_key: "2d7dbf6c25ea7bff194550d4597ea86f"
server_key_iv: "b81888dcbd7d879f0c37ad46c013ac3e"
server_key_version: "encryptionkey@0"
I20260811 07:42:47.638528 5358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "73a630174de649ae89e118ec7649a0e1"
format_stamp: "Formatted at 2026-08-11 07:42:47 on dist-test-slave-zkpd"
server_key: "2d7dbf6c25ea7bff194550d4597ea86f"
server_key_iv: "b81888dcbd7d879f0c37ad46c013ac3e"
server_key_version: "encryptionkey@0"
I20260811 07:42:47.642391 5358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 07:42:47.644857 5373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:47.646075 5358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:47.646209 5358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "73a630174de649ae89e118ec7649a0e1"
format_stamp: "Formatted at 2026-08-11 07:42:47 on dist-test-slave-zkpd"
server_key: "2d7dbf6c25ea7bff194550d4597ea86f"
server_key_iv: "b81888dcbd7d879f0c37ad46c013ac3e"
server_key_version: "encryptionkey@0"
I20260811 07:42:47.646319 5358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:47.658756 5358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:47.659417 5358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:47.659603 5358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:47.666544 5358 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:45015
I20260811 07:42:47.666553 5428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:45015 every 8 connection(s)
I20260811 07:42:47.667472 5358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:47.670648 5429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:47.673228 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 5358
I20260811 07:42:47.673367 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:42:47.673662 18723 external_mini_cluster.cc:1428] Setting key 075795460fc051d5336f7afe73548245
I20260811 07:42:47.676409 5429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1: Bootstrap starting.
I20260811 07:42:47.678809 5429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:47.679687 5429 log.cc:824] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:47.681811 5429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1: No bootstrap required, opened a new log
I20260811 07:42:47.682881 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.675074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.18.72.254:45015)
0811 07:42:47.675665 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:47.675681 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:47.675716 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:42:47.676387 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:47.676405 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:47.676832 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:47.677101 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:47.677437 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.678378 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.679633 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.680263 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.680444 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:47.681209 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:47.681213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:47.681226 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:47.681597 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:47.681650 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:47.681669 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:47.681744 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:47.681950 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 07:42:47.682230 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":136,"threads_started":1}
I20260811 07:42:47.685405 5429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73a630174de649ae89e118ec7649a0e1" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45015 } }
I20260811 07:42:47.685601 5429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:47.685680 5429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a630174de649ae89e118ec7649a0e1, State: Initialized, Role: FOLLOWER
I20260811 07:42:47.686136 5429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73a630174de649ae89e118ec7649a0e1" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45015 } }
I20260811 07:42:47.686300 5429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:47.686388 5429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:47.686494 5429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:47.687398 5429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73a630174de649ae89e118ec7649a0e1" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45015 } }
I20260811 07:42:47.687726 5429 leader_election.cc:302] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73a630174de649ae89e118ec7649a0e1; no voters:
I20260811 07:42:47.688031 5429 leader_election.cc:288] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:42:47.688129 5435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:47.688371 5435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [term 1 LEADER]: Becoming Leader. State: Replica: 73a630174de649ae89e118ec7649a0e1, State: Running, Role: LEADER
I20260811 07:42:47.688668 5435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73a630174de649ae89e118ec7649a0e1" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45015 } }
I20260811 07:42:47.689219 5429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:47.690299 5437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73a630174de649ae89e118ec7649a0e1. Latest consensus state: current_term: 1 leader_uuid: "73a630174de649ae89e118ec7649a0e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73a630174de649ae89e118ec7649a0e1" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45015 } } }
I20260811 07:42:47.690428 5437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:47.690707 5436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73a630174de649ae89e118ec7649a0e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73a630174de649ae89e118ec7649a0e1" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45015 } } }
I20260811 07:42:47.690817 5444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:47.690831 5436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:47.693620 5444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:47.698556 5444 catalog_manager.cc:1347] Generated new cluster ID: 3331929b2ae24373abb8e139797eb19d
I20260811 07:42:47.698617 5444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:47.710711 5444 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:42:47.711539 5444 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:47.719152 5444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1: Generated new TSK 0
I20260811 07:42:47.719787 5444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:42:47.726858 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45015
--builtin_ntp_servers=127.18.72.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:42:47.838716 5455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:47.838984 5455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:47.839051 5455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:47.842988 5455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:47.843055 5455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:47.843132 5455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:42:47.847745 5455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:47.848759 5455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:47.850132 5455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:47.859272 5463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:47.859315 5461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:47.859313 5466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:47.859488 5455 server_base.cc:1034] running on GCE node
I20260811 07:42:47.860015 5455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:47.860605 5455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:47.861788 5455 hybrid_clock.cc:648] HybridClock initialized: now 1786434167861771 us; error 28 us; skew 500 ppm
I20260811 07:42:47.863514 5455 webserver.cc:460] Webserver started at http://127.18.72.193:35545/ using document root <none> and password file <none>
I20260811 07:42:47.864131 5455 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:47.864217 5455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:47.864439 5455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:47.866364 5455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "c2587b590d194fe49550b838c90cbb87"
format_stamp: "Formatted at 2026-08-11 07:42:47 on dist-test-slave-zkpd"
server_key: "ac0086da02d62881fb056888f61c25da"
server_key_iv: "dbd7dfdeb4f53d98bad0f6ff1bbd6f75"
server_key_version: "encryptionkey@0"
I20260811 07:42:47.866915 5455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "c2587b590d194fe49550b838c90cbb87"
format_stamp: "Formatted at 2026-08-11 07:42:47 on dist-test-slave-zkpd"
server_key: "ac0086da02d62881fb056888f61c25da"
server_key_iv: "dbd7dfdeb4f53d98bad0f6ff1bbd6f75"
server_key_version: "encryptionkey@0"
I20260811 07:42:47.870441 5455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.005s
I20260811 07:42:47.872725 5473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:47.873915 5455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:47.874042 5455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "c2587b590d194fe49550b838c90cbb87"
format_stamp: "Formatted at 2026-08-11 07:42:47 on dist-test-slave-zkpd"
server_key: "ac0086da02d62881fb056888f61c25da"
server_key_iv: "dbd7dfdeb4f53d98bad0f6ff1bbd6f75"
server_key_version: "encryptionkey@0"
I20260811 07:42:47.874142 5455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:47.888562 5455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:47.889158 5455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:47.889338 5455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:47.889953 5455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:47.890920 5455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:47.890985 5455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:47.891048 5455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:47.891094 5455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:47.899514 5455 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:39359
I20260811 07:42:47.899557 5604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:39359 every 8 connection(s)
I20260811 07:42:47.900398 5455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:42:47.903728 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 5455
I20260811 07:42:47.903822 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:42:47.904145 18723 external_mini_cluster.cc:1428] Setting key 862aacf028fc02abd12f42a2dc360ff0
I20260811 07:42:47.906123 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45015
--builtin_ntp_servers=127.18.72.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:42:47.909773 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.902108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:35123 (local address 127.18.72.254:45015)
0811 07:42:47.902250 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:47.902255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:47.903279 (+ 1024us) server_negotiation.cc:407] Connection header received
0811 07:42:47.904810 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:47.904813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:47.904872 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:47.904953 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:47.906003 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.906589 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.907719 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.908043 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.908127 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:47.909347 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:47.909350 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:47.909352 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:47.909363 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:47.909382 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:47.909389 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:47.909446 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:47.909621 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 07:42:47.909668 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:42:47.911015 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:47.902522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45015 (local address 127.18.72.193:35123)
0811 07:42:47.903146 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:47.903183 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:47.904608 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:47.905147 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:47.905156 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:47.905256 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:47.905864 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:47.905879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.906712 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.906715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:47.907561 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:47.907570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:47.908193 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:47.908198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:47.908418 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:47.909058 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:47.909061 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:47.909115 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:47.909199 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:47.909470 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:47.909482 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:47.909491 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 07:42:47.909616 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 07:42:47.909991 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":112,"threads_started":1}
I20260811 07:42:47.912431 5605 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45015
I20260811 07:42:47.912688 5605 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:47.913323 5605 heartbeater.cc:511] Master 127.18.72.254:45015 requested a full tablet report, sending...
I20260811 07:42:47.914867 5391 ts_manager.cc:194] Registered new tserver with Master: c2587b590d194fe49550b838c90cbb87 (127.18.72.193:39359)
I20260811 07:42:47.916015 5391 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.193:35123
W20260811 07:42:48.019599 5608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:48.019832 5608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:48.019866 5608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:48.023869 5608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:48.023937 5608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:48.024026 5608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:42:48.028654 5608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:48.029672 5608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:48.031008 5608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:48.040570 5613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:48.040575 5616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:48.040575 5614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:48.040778 5608 server_base.cc:1034] running on GCE node
I20260811 07:42:48.041230 5608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:48.041802 5608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:48.042986 5608 hybrid_clock.cc:648] HybridClock initialized: now 1786434168042960 us; error 44 us; skew 500 ppm
I20260811 07:42:48.044734 5608 webserver.cc:460] Webserver started at http://127.18.72.194:43843/ using document root <none> and password file <none>
I20260811 07:42:48.045315 5608 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:48.045403 5608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:48.045625 5608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:48.047624 5608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "b5bc5f5e399544c2b50714748a4c05e9"
format_stamp: "Formatted at 2026-08-11 07:42:48 on dist-test-slave-zkpd"
server_key: "d0fd1c472ef530dc24683602f9b98956"
server_key_iv: "c44470261b300e0cae76ce7531eaf2b6"
server_key_version: "encryptionkey@0"
I20260811 07:42:48.048220 5608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "b5bc5f5e399544c2b50714748a4c05e9"
format_stamp: "Formatted at 2026-08-11 07:42:48 on dist-test-slave-zkpd"
server_key: "d0fd1c472ef530dc24683602f9b98956"
server_key_iv: "c44470261b300e0cae76ce7531eaf2b6"
server_key_version: "encryptionkey@0"
I20260811 07:42:48.052001 5608 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 07:42:48.054401 5622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:48.055590 5608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:48.055721 5608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "b5bc5f5e399544c2b50714748a4c05e9"
format_stamp: "Formatted at 2026-08-11 07:42:48 on dist-test-slave-zkpd"
server_key: "d0fd1c472ef530dc24683602f9b98956"
server_key_iv: "c44470261b300e0cae76ce7531eaf2b6"
server_key_version: "encryptionkey@0"
I20260811 07:42:48.055828 5608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:48.077013 5608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:48.077647 5608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:48.077852 5608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:48.078421 5608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:48.079326 5608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:48.079394 5608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:48.079459 5608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:48.079504 5608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:48.088452 5608 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:39083
I20260811 07:42:48.088490 5749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:39083 every 8 connection(s)
I20260811 07:42:48.089350 5608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:42:48.093982 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 5608
I20260811 07:42:48.094105 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:42:48.094453 18723 external_mini_cluster.cc:1428] Setting key fad7366d04df1af60e421c28d393a37c
I20260811 07:42:48.096606 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45015
--builtin_ntp_servers=127.18.72.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:42:48.099685 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.091157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:54719 (local address 127.18.72.254:45015)
0811 07:42:48.091303 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.091307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.092400 (+ 1093us) server_negotiation.cc:407] Connection header received
0811 07:42:48.093849 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.093851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.093901 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.093977 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:48.095145 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.095917 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.097601 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.097811 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.097901 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.099210 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:48.099213 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:48.099215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:48.099226 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:48.099245 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:48.099253 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:48.099334 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.099489 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.099550 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:42:48.100528 5752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.091586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45015 (local address 127.18.72.194:54719)
0811 07:42:48.092250 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:48.092299 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:48.093595 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:48.094197 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:48.094207 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:48.094333 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:48.094986 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.095007 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.096043 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.096046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.097481 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.097497 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.097973 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.097980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.098213 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.098952 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:48.098953 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:48.098992 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:48.099054 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:48.099346 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:48.099362 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:48.099368 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:42:48.099488 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 07:42:48.099762 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":110,"threads_started":1}
I20260811 07:42:48.101616 5750 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45015
I20260811 07:42:48.101855 5750 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:48.102373 5750 heartbeater.cc:511] Master 127.18.72.254:45015 requested a full tablet report, sending...
I20260811 07:42:48.103412 5391 ts_manager.cc:194] Registered new tserver with Master: b5bc5f5e399544c2b50714748a4c05e9 (127.18.72.194:39083)
I20260811 07:42:48.104292 5391 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.194:54719
W20260811 07:42:48.209533 5753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:48.209769 5753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:48.209803 5753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:48.213572 5753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:48.213613 5753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:48.213661 5753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:42:48.218106 5753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:48.219041 5753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:48.220373 5753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:48.229430 5760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:48.229430 5759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:48.229558 5753 server_base.cc:1034] running on GCE node
W20260811 07:42:48.229686 5762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:48.230096 5753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:48.230638 5753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:48.231787 5753 hybrid_clock.cc:648] HybridClock initialized: now 1786434168231759 us; error 42 us; skew 500 ppm
I20260811 07:42:48.233474 5753 webserver.cc:460] Webserver started at http://127.18.72.195:37343/ using document root <none> and password file <none>
I20260811 07:42:48.234282 5753 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:48.234390 5753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:48.234609 5753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:48.236726 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "077307753daa4e2a8a667170f1a4f92d"
format_stamp: "Formatted at 2026-08-11 07:42:48 on dist-test-slave-zkpd"
server_key: "f208adee6d5b892e5a75a1106ff6472a"
server_key_iv: "4100a43e46754cbf73825a3c27e01346"
server_key_version: "encryptionkey@0"
I20260811 07:42:48.237285 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "077307753daa4e2a8a667170f1a4f92d"
format_stamp: "Formatted at 2026-08-11 07:42:48 on dist-test-slave-zkpd"
server_key: "f208adee6d5b892e5a75a1106ff6472a"
server_key_iv: "4100a43e46754cbf73825a3c27e01346"
server_key_version: "encryptionkey@0"
I20260811 07:42:48.241155 5753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 07:42:48.243484 5769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:48.244701 5753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:48.244854 5753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "077307753daa4e2a8a667170f1a4f92d"
format_stamp: "Formatted at 2026-08-11 07:42:48 on dist-test-slave-zkpd"
server_key: "f208adee6d5b892e5a75a1106ff6472a"
server_key_iv: "4100a43e46754cbf73825a3c27e01346"
server_key_version: "encryptionkey@0"
I20260811 07:42:48.244962 5753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:48.261152 5753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:48.261771 5753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:48.261953 5753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:48.262511 5753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:48.263424 5753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:48.263494 5753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:48.263559 5753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:48.263604 5753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:48.272667 5753 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:34091
I20260811 07:42:48.272707 5894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:34091 every 8 connection(s)
I20260811 07:42:48.273574 5753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:42:48.282943 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.275378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:54795 (local address 127.18.72.254:45015)
0811 07:42:48.275509 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.275513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.276609 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 07:42:48.278183 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.278186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.278283 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.278375 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:48.279254 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.279792 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.281016 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.281244 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.281357 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.282443 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:48.282446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:48.282449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:48.282467 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:48.282486 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:48.282494 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:48.282574 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.282726 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.282800 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:42:48.283823 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.275774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:45015 (local address 127.18.72.195:54795)
0811 07:42:48.276463 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:48.276504 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:48.277967 (+ 1463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:48.278546 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:48.278553 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:48.278635 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:48.279114 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.279125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.279947 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.279950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.280885 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.280895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.281402 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.281406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.281571 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.282178 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:48.282180 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:48.282218 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:48.282287 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:48.282576 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:48.282589 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:48.282596 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:42:48.282725 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 07:42:48.282999 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":142,"threads_started":1}
I20260811 07:42:48.283931 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 5753
I20260811 07:42:48.284031 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:42:48.284380 18723 external_mini_cluster.cc:1428] Setting key d82287c44771a304705f8b3a45dc6d00
I20260811 07:42:48.285086 5895 heartbeater.cc:347] Connected to a master server at 127.18.72.254:45015
I20260811 07:42:48.285358 5895 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:48.286023 5895 heartbeater.cc:511] Master 127.18.72.254:45015 requested a full tablet report, sending...
I20260811 07:42:48.287081 5391 ts_manager.cc:194] Registered new tserver with Master: 077307753daa4e2a8a667170f1a4f92d (127.18.72.195:34091)
I20260811 07:42:48.287599 5391 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.195:54795
I20260811 07:42:48.298527 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:42:48.306806 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.300883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.18.72.254:45015)
0811 07:42:48.301054 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.301061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.301899 (+ 838us) server_negotiation.cc:407] Connection header received
0811 07:42:48.302085 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.302088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.302160 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.302265 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:48.302613 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.303507 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.304907 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.305290 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.305392 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.306313 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:48.306316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:48.306319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:48.306332 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:48.306385 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:48.306396 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:48.306465 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.306563 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.306632 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:42:48.310111 5391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:42:48.313323 5391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:42:48.331008 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.324357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:39083 (local address 127.0.0.1:53562)
0811 07:42:48.325122 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:48.325137 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:48.325223 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:48.325820 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:48.325826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:48.325876 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:48.326336 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.326344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.327624 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.327627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.329217 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.329226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.330335 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.330340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.330497 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.330572 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 07:42:48.330695 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 07:42:48.330844 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":86,"spinlock_wait_cycles":34432,"thread_start_us":93,"threads_started":1}
I20260811 07:42:48.331705 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.324645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53562 (local address 127.18.72.194:39083)
0811 07:42:48.324995 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.325006 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.325328 (+ 322us) server_negotiation.cc:407] Connection header received
0811 07:42:48.325404 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.325408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.325552 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.325674 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:48.326440 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.327485 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.329352 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.330203 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.330314 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.330510 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.331359 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.331485 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":123,"threads_started":1}
I20260811 07:42:48.332814 5908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.323213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:34091 (local address 127.0.0.1:46722)
0811 07:42:48.323725 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:48.323778 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:48.323956 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:48.325820 (+ 1864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:48.325829 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:48.325871 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:48.326130 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.326138 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.329295 (+ 3157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.329299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.330540 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.330553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.332435 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.332439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.332519 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.332543 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 07:42:48.332611 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 07:42:48.332677 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":70,"spinlock_wait_cycles":9728,"thread_start_us":83,"threads_started":1}
I20260811 07:42:48.333361 5910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.323960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46722 (local address 127.18.72.195:34091)
0811 07:42:48.324391 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.324405 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.324432 (+ 27us) server_negotiation.cc:407] Connection header received
0811 07:42:48.324954 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.324962 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.325578 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.325997 (+ 419us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:48.326235 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.327489 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.330662 (+ 3173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.331460 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.331560 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.331731 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.333088 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.333192 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":104,"threads_started":1}
I20260811 07:42:48.335361 5677 tablet_service.cc:1467] Processing CreateTablet for tablet 32093853d8cd46779fd35e4db852f86f (DEFAULT_TABLE table=test-table [id=e55d5b6e8ceb4605ae54041e6ca264d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:48.336670 5677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32093853d8cd46779fd35e4db852f86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:48.338699 5821 tablet_service.cc:1467] Processing CreateTablet for tablet 32093853d8cd46779fd35e4db852f86f (DEFAULT_TABLE table=test-table [id=e55d5b6e8ceb4605ae54041e6ca264d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:48.339746 5821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32093853d8cd46779fd35e4db852f86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:48.344509 5914 tablet_bootstrap.cc:492] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: Bootstrap starting.
I20260811 07:42:48.347150 5915 tablet_bootstrap.cc:492] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: Bootstrap starting.
I20260811 07:42:48.348316 5914 tablet_bootstrap.cc:654] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:48.349587 5915 tablet_bootstrap.cc:654] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:48.349472 5914 log.cc:824] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:48.350214 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.323346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39359 (local address 127.0.0.1:42536)
0811 07:42:48.324583 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:48.324596 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:48.324698 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:48.325842 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:48.325844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:48.325871 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:48.326136 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.326142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.329224 (+ 3082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.329226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.332258 (+ 3032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.332269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.349243 (+ 16974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.349248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.349448 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.349507 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 07:42:48.349571 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 07:42:48.350065 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1173,"mutex_wait_us":116,"spinlock_wait_cycles":9984,"thread_start_us":89,"threads_started":1}
I20260811 07:42:48.350505 5915 log.cc:824] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:48.351063 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.323489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.18.72.193:39359)
0811 07:42:48.324449 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.324464 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.324690 (+ 226us) server_negotiation.cc:407] Connection header received
0811 07:42:48.325263 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.325272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.325483 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.325652 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:48.327785 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.329102 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.332407 (+ 3305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.349063 (+ 16656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.350426 (+ 1363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.350740 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.350800 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.350916 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":119,"threads_started":1}
I20260811 07:42:48.352344 5915 tablet_bootstrap.cc:492] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: No bootstrap required, opened a new log
I20260811 07:42:48.352545 5915 ts_tablet_manager.cc:1397] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 07:42:48.352706 5914 tablet_bootstrap.cc:492] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: No bootstrap required, opened a new log
I20260811 07:42:48.352900 5914 ts_tablet_manager.cc:1397] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 07:42:48.353968 5530 tablet_service.cc:1467] Processing CreateTablet for tablet 32093853d8cd46779fd35e4db852f86f (DEFAULT_TABLE table=test-table [id=e55d5b6e8ceb4605ae54041e6ca264d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:48.355018 5530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32093853d8cd46779fd35e4db852f86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:48.355876 5915 raft_consensus.cc:348] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.356168 5915 raft_consensus.cc:374] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:48.356242 5915 raft_consensus.cc:729] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 077307753daa4e2a8a667170f1a4f92d, State: Initialized, Role: FOLLOWER
I20260811 07:42:48.356675 5915 consensus_queue.cc:260] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.356992 5914 raft_consensus.cc:348] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.357303 5914 raft_consensus.cc:374] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:48.357309 5895 heartbeater.cc:503] Master 127.18.72.254:45015 was elected leader, sending a full tablet report...
I20260811 07:42:48.357416 5914 raft_consensus.cc:729] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bc5f5e399544c2b50714748a4c05e9, State: Initialized, Role: FOLLOWER
I20260811 07:42:48.357656 5915 ts_tablet_manager.cc:1428] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260811 07:42:48.357961 5914 consensus_queue.cc:260] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.358772 5750 heartbeater.cc:503] Master 127.18.72.254:45015 was elected leader, sending a full tablet report...
I20260811 07:42:48.358994 5914 ts_tablet_manager.cc:1428] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 07:42:48.363030 5920 tablet_bootstrap.cc:492] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87: Bootstrap starting.
I20260811 07:42:48.363247 5918 raft_consensus.cc:482] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:48.363447 5918 raft_consensus.cc:504] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.364610 5918 leader_election.cc:288] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2587b590d194fe49550b838c90cbb87 (127.18.72.193:39359), b5bc5f5e399544c2b50714748a4c05e9 (127.18.72.194:39083)
I20260811 07:42:48.365325 5920 tablet_bootstrap.cc:654] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:48.366250 5920 log.cc:824] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:48.368638 5920 tablet_bootstrap.cc:492] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87: No bootstrap required, opened a new log
I20260811 07:42:48.368840 5920 ts_tablet_manager.cc:1397] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 07:42:48.369480 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.365425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:52853 (local address 127.18.72.193:39359)
0811 07:42:48.365532 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.365537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.365551 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:42:48.365593 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.365596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.365648 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.365733 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:48.366001 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.366705 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.368325 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.368940 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.369071 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.369115 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.369306 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.369374 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:42:48.369482 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.364812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39359 (local address 127.18.72.195:52853)
0811 07:42:48.364941 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:48.364957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:48.365048 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:48.365757 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:48.365760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:48.365778 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:48.365895 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.365903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.366863 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.366870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.368180 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.368192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.369056 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.369060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.369165 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.369190 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 07:42:48.369277 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:42:48.369358 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 07:42:48.369532 5921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.365074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:39083 (local address 127.18.72.195:37931)
0811 07:42:48.365408 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:48.365422 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:48.365513 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:48.366242 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:48.366245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:48.366279 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:48.366392 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.366398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.367204 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.367208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.368363 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:48.368371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.369196 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.369200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:48.369296 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.369315 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 07:42:48.369376 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 07:42:48.369428 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260811 07:42:48.369844 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.365892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:37931 (local address 127.18.72.194:39083)
0811 07:42:48.366021 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.366026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.366041 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:42:48.366088 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.366093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.366147 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.366225 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:48.366516 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.367089 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.368471 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.368979 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.369093 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.369144 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.369630 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.369721 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:42:48.370354 5701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32093853d8cd46779fd35e4db852f86f" candidate_uuid: "077307753daa4e2a8a667170f1a4f92d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bc5f5e399544c2b50714748a4c05e9" is_pre_election: true
I20260811 07:42:48.370616 5701 raft_consensus.cc:2454] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 077307753daa4e2a8a667170f1a4f92d in term 0.
I20260811 07:42:48.370116 5551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32093853d8cd46779fd35e4db852f86f" candidate_uuid: "077307753daa4e2a8a667170f1a4f92d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2587b590d194fe49550b838c90cbb87" is_pre_election: true
I20260811 07:42:48.371090 5771 leader_election.cc:302] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 077307753daa4e2a8a667170f1a4f92d, b5bc5f5e399544c2b50714748a4c05e9; no voters:
I20260811 07:42:48.371538 5918 raft_consensus.cc:2781] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:42:48.371655 5918 raft_consensus.cc:482] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:42:48.371735 5918 raft_consensus.cc:3037] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 0 FOLLOWER]: Advancing to term 1
W20260811 07:42:48.371718 5771 leader_election.cc:341] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2587b590d194fe49550b838c90cbb87 (127.18.72.193:39359): Illegal state: must be running to vote when last-logged opid is not known
I20260811 07:42:48.372885 5920 raft_consensus.cc:348] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.373037 5918 raft_consensus.cc:504] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.373121 5920 raft_consensus.cc:374] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:48.373168 5920 raft_consensus.cc:729] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2587b590d194fe49550b838c90cbb87, State: Initialized, Role: FOLLOWER
I20260811 07:42:48.373389 5918 leader_election.cc:288] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [CANDIDATE]: Term 1 election: Requested vote from peers c2587b590d194fe49550b838c90cbb87 (127.18.72.193:39359), b5bc5f5e399544c2b50714748a4c05e9 (127.18.72.194:39083)
I20260811 07:42:48.373601 5920 consensus_queue.cc:260] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.373842 5701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32093853d8cd46779fd35e4db852f86f" candidate_uuid: "077307753daa4e2a8a667170f1a4f92d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bc5f5e399544c2b50714748a4c05e9"
I20260811 07:42:48.373966 5701 raft_consensus.cc:3037] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:48.374266 5605 heartbeater.cc:503] Master 127.18.72.254:45015 was elected leader, sending a full tablet report...
I20260811 07:42:48.374624 5920 ts_tablet_manager.cc:1428] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:42:48.375257 5701 raft_consensus.cc:2454] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 077307753daa4e2a8a667170f1a4f92d in term 1.
I20260811 07:42:48.375561 5551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32093853d8cd46779fd35e4db852f86f" candidate_uuid: "077307753daa4e2a8a667170f1a4f92d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2587b590d194fe49550b838c90cbb87"
I20260811 07:42:48.375713 5551 raft_consensus.cc:3037] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:48.376132 5771 leader_election.cc:302] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 077307753daa4e2a8a667170f1a4f92d, b5bc5f5e399544c2b50714748a4c05e9; no voters:
I20260811 07:42:48.376868 5551 raft_consensus.cc:2454] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 077307753daa4e2a8a667170f1a4f92d in term 1.
I20260811 07:42:48.377110 5918 raft_consensus.cc:2781] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:48.377373 5918 raft_consensus.cc:686] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 1 LEADER]: Becoming Leader. State: Replica: 077307753daa4e2a8a667170f1a4f92d, State: Running, Role: LEADER
I20260811 07:42:48.377689 5918 consensus_queue.cc:237] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } }
I20260811 07:42:48.380652 5390 catalog_manager.cc:5495] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d reported cstate change: term changed from 0 to 1, leader changed from <none> to 077307753daa4e2a8a667170f1a4f92d (127.18.72.195). New cstate: current_term: 1 leader_uuid: "077307753daa4e2a8a667170f1a4f92d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "077307753daa4e2a8a667170f1a4f92d" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 34091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 } health_report { overall_health: UNKNOWN } } }
I20260811 07:42:48.396832 5910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.393281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.18.72.195:34091)
0811 07:42:48.393397 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:48.393402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:48.393443 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:42:48.393622 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:48.393625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:48.393683 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:48.393809 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:42:48.394034 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.394535 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.395402 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:48.395620 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:48.395725 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:48.396062 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:42:48.396437 (+ 375us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:42:48.396559 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:48.396620 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 07:42:48.396668 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 07:42:48.401480 5606 tablet.cc:2369] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:42:48.402443 5551 raft_consensus.cc:1261] T 32093853d8cd46779fd35e4db852f86f P c2587b590d194fe49550b838c90cbb87 [term 1 FOLLOWER]: Refusing update from remote peer 077307753daa4e2a8a667170f1a4f92d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:48.402504 5701 raft_consensus.cc:1261] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 1 FOLLOWER]: Refusing update from remote peer 077307753daa4e2a8a667170f1a4f92d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:48.403097 5924 consensus_queue.cc:1035] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2587b590d194fe49550b838c90cbb87" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 39359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:48.403275 5918 consensus_queue.cc:1035] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bc5f5e399544c2b50714748a4c05e9" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:48.411190 5929 mvcc.cc:205] Tried to move back new op lower bound from 7317234353769312256 to 7317234353677619200. Current Snapshot: MvccSnapshot[applied={T|T < 7317234353769312256}]
I20260811 07:42:48.413164 5928 mvcc.cc:205] Tried to move back new op lower bound from 7317234353769312256 to 7317234353677619200. Current Snapshot: MvccSnapshot[applied={T|T < 7317234353769312256}]
I20260811 07:42:48.419548 5391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:48.419740 5391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:48.422163 5391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73a630174de649ae89e118ec7649a0e1: Sending DeleteTablet for 3 replicas of tablet 32093853d8cd46779fd35e4db852f86f
I20260811 07:42:48.422874 5821 tablet_service.cc:1514] Processing DeleteTablet for tablet 32093853d8cd46779fd35e4db852f86f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:48 UTC) from {username='slave'} at 127.0.0.1:46722
I20260811 07:42:48.423007 5677 tablet_service.cc:1514] Processing DeleteTablet for tablet 32093853d8cd46779fd35e4db852f86f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:48 UTC) from {username='slave'} at 127.0.0.1:53562
I20260811 07:42:48.423362 5938 tablet_replica.cc:330] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: stopping tablet replica
I20260811 07:42:48.423455 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 5455
I20260811 07:42:48.423640 5938 raft_consensus.cc:2229] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:42:48.423918 5938 raft_consensus.cc:2258] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:48.424340 5940 tablet_replica.cc:330] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: stopping tablet replica
I20260811 07:42:48.424628 5940 raft_consensus.cc:2229] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:48.424916 5940 raft_consensus.cc:2258] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:48.426131 5938 ts_tablet_manager.cc:1905] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:48.426231 5940 ts_tablet_manager.cc:1905] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:48.428858 5940 ts_tablet_manager.cc:1918] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:48.428850 5938 ts_tablet_manager.cc:1918] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:48.428946 5940 log.cc:1196] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/32093853d8cd46779fd35e4db852f86f
I20260811 07:42:48.428946 5938 log.cc:1196] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/32093853d8cd46779fd35e4db852f86f
I20260811 07:42:48.429260 5938 ts_tablet_manager.cc:1939] T 32093853d8cd46779fd35e4db852f86f P 077307753daa4e2a8a667170f1a4f92d: Deleting consensus metadata
I20260811 07:42:48.429260 5940 ts_tablet_manager.cc:1939] T 32093853d8cd46779fd35e4db852f86f P b5bc5f5e399544c2b50714748a4c05e9: Deleting consensus metadata
I20260811 07:42:48.430356 5374 catalog_manager.cc:4841] TS 077307753daa4e2a8a667170f1a4f92d (127.18.72.195:34091): tablet 32093853d8cd46779fd35e4db852f86f (table test-table [id=e55d5b6e8ceb4605ae54041e6ca264d5]) successfully deleted
I20260811 07:42:48.430540 5375 catalog_manager.cc:4841] TS b5bc5f5e399544c2b50714748a4c05e9 (127.18.72.194:39083): tablet 32093853d8cd46779fd35e4db852f86f (table test-table [id=e55d5b6e8ceb4605ae54041e6ca264d5]) successfully deleted
W20260811 07:42:48.441530 5375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:39359 (error 108)
I20260811 07:42:48.442598 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:48.442287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:39359 (local address 127.0.0.1:42542)
0811 07:42:48.442414 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:48.442494 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 07:42:48.442701 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 5608
W20260811 07:42:48.442817 5375 catalog_manager.cc:4584] TS c2587b590d194fe49550b838c90cbb87 (127.18.72.193:39359): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 32093853d8cd46779fd35e4db852f86f: Network error: Client connection negotiation failed: client connection to 127.18.72.193:39359: connect: Connection refused (error 111)
I20260811 07:42:48.453308 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 5753
I20260811 07:42:48.461120 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 5358
2026-08-11T07:42:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (997 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:42:48 dist-test-slave-zkpd krb5kdc[5944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:42:48 dist-test-slave-zkpd krb5kdc[5944](info): set up 2 sockets
Aug 11 07:42:48 dist-test-slave-zkpd krb5kdc[5944](info): commencing operation
krb5kdc: starting...
W20260811 07:42:49.560853 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:42:49 dist-test-slave-zkpd krb5kdc[5944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:42:49.682891 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:39453
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:39453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:49.794159 5959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:49.794428 5959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:49.794505 5959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:49.798503 5959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:49.798580 5959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:49.798619 5959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:49.798671 5959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:49.798712 5959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:49.803782 5959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:39453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:39453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:49.804752 5959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:49.805704 5959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:49.813771 5965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:49.813838 5968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:49.813864 5964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:49.813949 5959 server_base.cc:1034] running on GCE node
I20260811 07:42:49.814599 5959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:49.815616 5959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:49.816809 5959 hybrid_clock.cc:648] HybridClock initialized: now 1786434169816766 us; error 56 us; skew 500 ppm
Aug 11 07:42:49 dist-test-slave-zkpd krb5kdc[5944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434169, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:49.823415 5959 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:42:49.824307 5959 webserver.cc:460] Webserver started at http://127.18.72.254:36931/ using document root <none> and password file <none>
I20260811 07:42:49.824853 5959 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:49.824937 5959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:49.825158 5959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:49.827145 5959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "86463df27d0e40b6abdab80c73c12a7b"
format_stamp: "Formatted at 2026-08-11 07:42:49 on dist-test-slave-zkpd"
server_key: "2a29dc24c8e728149d0543bc188dcbea"
server_key_iv: "47e6de6d0a78e415082fce1a484b4cc2"
server_key_version: "encryptionkey@0"
I20260811 07:42:49.827698 5959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "86463df27d0e40b6abdab80c73c12a7b"
format_stamp: "Formatted at 2026-08-11 07:42:49 on dist-test-slave-zkpd"
server_key: "2a29dc24c8e728149d0543bc188dcbea"
server_key_iv: "47e6de6d0a78e415082fce1a484b4cc2"
server_key_version: "encryptionkey@0"
I20260811 07:42:49.831446 5959 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:42:49.833842 5977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:49.835016 5959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:49.835150 5959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "86463df27d0e40b6abdab80c73c12a7b"
format_stamp: "Formatted at 2026-08-11 07:42:49 on dist-test-slave-zkpd"
server_key: "2a29dc24c8e728149d0543bc188dcbea"
server_key_iv: "47e6de6d0a78e415082fce1a484b4cc2"
server_key_version: "encryptionkey@0"
I20260811 07:42:49.835258 5959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:49.857451 5959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:49.860191 5959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:49.860378 5959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:49.867188 5959 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:39453
I20260811 07:42:49.867206 6034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:39453 every 8 connection(s)
I20260811 07:42:49.868140 5959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:49.869556 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 5959
I20260811 07:42:49.869689 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:42:49.869976 18723 external_mini_cluster.cc:1428] Setting key 0003f60ee2cd023eb72f699632a7e1c0
I20260811 07:42:49.871053 6035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:49.876469 6035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b: Bootstrap starting.
Aug 11 07:42:49 dist-test-slave-zkpd krb5kdc[5944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:49.878908 6035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:49.879753 6035 log.cc:824] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:49.881884 6035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b: No bootstrap required, opened a new log
I20260811 07:42:49.883723 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:49.871340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.18.72.254:39453)
0811 07:42:49.871753 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:49.871767 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:49.871801 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:42:49.872473 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:49.872489 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:49.872789 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:49.873134 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:49.873955 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:49.874915 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:49.875768 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:49.876285 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:49.876458 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:49.879115 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:49.879121 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:49.879134 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:49.879160 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:49.880987 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:49.881447 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:49.881452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:49.881456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:49.881555 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:49.881867 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:49.881870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:49.881870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:49.882301 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:49.882438 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:49.882746 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 07:42:49.883045 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":140,"threads_started":1}
I20260811 07:42:49.886049 6035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86463df27d0e40b6abdab80c73c12a7b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39453 } }
I20260811 07:42:49.886255 6035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:49.886322 6035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86463df27d0e40b6abdab80c73c12a7b, State: Initialized, Role: FOLLOWER
I20260811 07:42:49.886725 6035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86463df27d0e40b6abdab80c73c12a7b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39453 } }
I20260811 07:42:49.886864 6035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:49.886945 6035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:49.887056 6035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:49.887882 6035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86463df27d0e40b6abdab80c73c12a7b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39453 } }
I20260811 07:42:49.888264 6035 leader_election.cc:302] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86463df27d0e40b6abdab80c73c12a7b; no voters:
I20260811 07:42:49.888586 6035 leader_election.cc:288] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:42:49.888702 6040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:49.888942 6040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [term 1 LEADER]: Becoming Leader. State: Replica: 86463df27d0e40b6abdab80c73c12a7b, State: Running, Role: LEADER
I20260811 07:42:49.889273 6040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86463df27d0e40b6abdab80c73c12a7b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39453 } }
I20260811 07:42:49.889761 6035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:49.891026 6042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86463df27d0e40b6abdab80c73c12a7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86463df27d0e40b6abdab80c73c12a7b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39453 } } }
I20260811 07:42:49.891034 6043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86463df27d0e40b6abdab80c73c12a7b. Latest consensus state: current_term: 1 leader_uuid: "86463df27d0e40b6abdab80c73c12a7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86463df27d0e40b6abdab80c73c12a7b" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39453 } } }
I20260811 07:42:49.891220 6043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:49.891770 6051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:49.891522 6042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:49.894727 6051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:49.899703 6051 catalog_manager.cc:1347] Generated new cluster ID: f187f4f5bd1d4f0db4456e10f5e3dc5e
I20260811 07:42:49.899765 6051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:49.906005 6051 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:42:49.906841 6051 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:49.912565 6051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b: Generated new TSK 0
I20260811 07:42:49.913168 6051 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:42:49.941129 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:39453
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:50.055931 6062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:50.056221 6062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:50.056301 6062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:50.060209 6062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:50.060256 6062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:50.060352 6062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:42:50.065302 6062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:39453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:50.066268 6062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:50.067272 6062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:50.076303 6069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:50.076316 6072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:50.076303 6070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:50.076524 6062 server_base.cc:1034] running on GCE node
I20260811 07:42:50.077041 6062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:50.077623 6062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:50.078800 6062 hybrid_clock.cc:648] HybridClock initialized: now 1786434170078771 us; error 48 us; skew 500 ppm
Aug 11 07:42:50 dist-test-slave-zkpd krb5kdc[5944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434170, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:50.081422 6062 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:42:50.082249 6062 webserver.cc:460] Webserver started at http://127.18.72.193:39177/ using document root <none> and password file <none>
I20260811 07:42:50.082779 6062 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:50.082875 6062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:50.083128 6062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:50.085196 6062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "ebc7118365ec412e8a6118e6b3bd1574"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "bfdde6e3c7496cf55e9cab9c3998a029"
server_key_iv: "5ac5f61bb6d54e3719d67e0bc47a6ae3"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.085764 6062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "ebc7118365ec412e8a6118e6b3bd1574"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "bfdde6e3c7496cf55e9cab9c3998a029"
server_key_iv: "5ac5f61bb6d54e3719d67e0bc47a6ae3"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.089408 6062 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 07:42:50.092068 6079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.093261 6062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:42:50.093396 6062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "ebc7118365ec412e8a6118e6b3bd1574"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "bfdde6e3c7496cf55e9cab9c3998a029"
server_key_iv: "5ac5f61bb6d54e3719d67e0bc47a6ae3"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.093508 6062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:50.106971 6062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:50.109737 6062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:50.109941 6062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:50.110508 6062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:50.111336 6062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:50.111403 6062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.111469 6062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:50.111517 6062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.120764 6062 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:37701
I20260811 07:42:50.120791 6205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:37701 every 8 connection(s)
I20260811 07:42:50.121608 6062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:42:50.127287 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 6062
I20260811 07:42:50.127449 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:42:50.127750 18723 external_mini_cluster.cc:1428] Setting key 95f7ccc9ed6346df74b681b613b28a03
Aug 11 07:42:50 dist-test-slave-zkpd krb5kdc[5944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434170, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:50.135762 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.123311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:48977 (local address 127.18.72.254:39453)
0811 07:42:50.123438 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.123442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.124262 (+ 820us) server_negotiation.cc:407] Connection header received
0811 07:42:50.125633 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.125637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.125696 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.125809 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:50.126922 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.127526 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.128886 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.129251 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.129368 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.132023 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:50.132026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:50.132028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:50.132053 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:50.133490 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:50.134146 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:50.134148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:50.134150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:50.134225 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:50.134638 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:50.134641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:50.134642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:50.134817 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:50.134894 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.135537 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.135644 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:42:50.137022 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.123647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:39453 (local address 127.18.72.193:48977)
0811 07:42:50.124115 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.124156 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:50.125425 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:50.125971 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:50.125978 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:50.126314 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:50.126769 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.126785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.127655 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.127663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.128758 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.128769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.129403 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.129408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.129575 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.130152 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:50.130168 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:50.131835 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:50.133610 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:50.133619 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:50.133636 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:50.134032 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:50.134318 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:50.134324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:50.134329 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:50.134530 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:50.134927 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:50.134934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:50.135264 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 07:42:50.135489 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 07:42:50.135871 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":110,"threads_started":1}
I20260811 07:42:50.138311 6206 heartbeater.cc:347] Connected to a master server at 127.18.72.254:39453
I20260811 07:42:50.138554 6206 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
I20260811 07:42:50.139118 6206 heartbeater.cc:511] Master 127.18.72.254:39453 requested a full tablet report, sending...
I20260811 07:42:50.140705 5995 ts_manager.cc:194] Registered new tserver with Master: ebc7118365ec412e8a6118e6b3bd1574 (127.18.72.193:37701)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
I20260811 07:42:50.142169 5995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:48977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:42:50.156668 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:39453
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:50.274197 6211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:50.274427 6211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:50.274492 6211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:50.278471 6211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:50.278524 6211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:50.278625 6211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:42:50.283685 6211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:39453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:50.284776 6211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:50.285784 6211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:50.295666 6220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:50.295653 6216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:50.295648 6217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:50.296060 6211 server_base.cc:1034] running on GCE node
I20260811 07:42:50.296514 6211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:50.297164 6211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:50.298345 6211 hybrid_clock.cc:648] HybridClock initialized: now 1786434170298310 us; error 67 us; skew 500 ppm
Aug 11 07:42:50 dist-test-slave-zkpd krb5kdc[5944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434170, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:50.301258 6211 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:42:50.302100 6211 webserver.cc:460] Webserver started at http://127.18.72.194:35575/ using document root <none> and password file <none>
I20260811 07:42:50.302682 6211 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:50.302742 6211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:50.302923 6211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:50.305059 6211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "4daea56190354526af4ab10348c288ce"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "b94e0465ece9b6d22534b2543211ca1b"
server_key_iv: "51f6285d5fdd19574c77ba66ba9544b3"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.305795 6211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "4daea56190354526af4ab10348c288ce"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "b94e0465ece9b6d22534b2543211ca1b"
server_key_iv: "51f6285d5fdd19574c77ba66ba9544b3"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.309665 6211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 07:42:50.312121 6227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.313359 6211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:50.313478 6211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "4daea56190354526af4ab10348c288ce"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "b94e0465ece9b6d22534b2543211ca1b"
server_key_iv: "51f6285d5fdd19574c77ba66ba9544b3"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.313578 6211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:50.345870 6211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:50.349273 6211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:50.349479 6211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:50.350099 6211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:50.350925 6211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:50.350972 6211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.351050 6211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:50.351082 6211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.359722 6211 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:44567
I20260811 07:42:50.359742 6351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:44567 every 8 connection(s)
I20260811 07:42:50.360606 6211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:42:50.363081 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 6211
I20260811 07:42:50.363233 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:42:50.363600 18723 external_mini_cluster.cc:1428] Setting key 93642e4fc6c39cf80f1e987e183be031
Aug 11 07:42:50 dist-test-slave-zkpd krb5kdc[5944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434170, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:42:50.376077 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.362277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:55053 (local address 127.18.72.254:39453)
0811 07:42:50.362402 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.362405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.363257 (+ 852us) server_negotiation.cc:407] Connection header received
0811 07:42:50.364969 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.364972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.365031 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.365109 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:50.366280 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.367128 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.368958 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.369178 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.369268 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.372167 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:50.372173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:50.372179 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:50.372215 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:50.373894 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:50.374414 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:50.374420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:50.374424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:50.374527 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:50.374861 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:50.374868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:50.374870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:50.375113 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:50.375230 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.375689 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.375847 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:42:50.376780 6354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.362593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:39453 (local address 127.18.72.194:55053)
0811 07:42:50.363086 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.363147 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:50.364707 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:50.365323 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:50.365331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:50.365665 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:50.366137 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.366149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.367263 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.367268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.368793 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.368809 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.369338 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.369344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.369574 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.370187 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:50.370210 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:50.371998 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:50.374032 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:50.374037 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:50.374049 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:50.374297 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:50.374625 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:50.374627 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:50.374628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:50.374752 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:50.375216 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:50.375221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:50.375444 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 07:42:50.375587 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 07:42:50.375887 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":103,"threads_started":1}
I20260811 07:42:50.377950 6352 heartbeater.cc:347] Connected to a master server at 127.18.72.254:39453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
I20260811 07:42:50.378186 6352 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:50.378719 6352 heartbeater.cc:511] Master 127.18.72.254:39453 requested a full tablet report, sending...
I20260811 07:42:50.379927 5995 ts_manager.cc:194] Registered new tserver with Master: 4daea56190354526af4ab10348c288ce (127.18.72.194:44567)
I20260811 07:42:50.380497 5995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:55053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:42:50.392767 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:39453
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:50.504741 6357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:50.504969 6357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:50.505008 6357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:50.508960 6357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:50.509023 6357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:50.509076 6357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:42:50.514120 6357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:39453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:50.515017 6357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:50.515992 6357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:50.525712 6365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:50.525767 6357 server_base.cc:1034] running on GCE node
W20260811 07:42:50.525674 6362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:50.525676 6363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:50.526357 6357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:50.526943 6357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:50.528103 6357 hybrid_clock.cc:648] HybridClock initialized: now 1786434170528072 us; error 42 us; skew 500 ppm
Aug 11 07:42:50 dist-test-slave-zkpd krb5kdc[5944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434170, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:50.530701 6357 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:42:50.531527 6357 webserver.cc:460] Webserver started at http://127.18.72.195:38389/ using document root <none> and password file <none>
I20260811 07:42:50.532124 6357 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:50.532208 6357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:50.532449 6357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:50.534368 6357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "a436a84c5e404443ae058bb3be0a00ee"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "a15c9299160044bd6312779625efa3d1"
server_key_iv: "bbe7cfd8624d54f8335b363a98ff1427"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.534919 6357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "a436a84c5e404443ae058bb3be0a00ee"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "a15c9299160044bd6312779625efa3d1"
server_key_iv: "bbe7cfd8624d54f8335b363a98ff1427"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.538554 6357 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 07:42:50.540901 6373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.542080 6357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:50.542202 6357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "a436a84c5e404443ae058bb3be0a00ee"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "a15c9299160044bd6312779625efa3d1"
server_key_iv: "bbe7cfd8624d54f8335b363a98ff1427"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.542302 6357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:50.570732 6357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:50.573592 6357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:50.573801 6357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:50.574383 6357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:50.575179 6357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:50.575251 6357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.575320 6357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:50.575371 6357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.584395 6357 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:35743
I20260811 07:42:50.584419 6493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:35743 every 8 connection(s)
I20260811 07:42:50.585256 6357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:42:50.588894 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 6357
I20260811 07:42:50.589042 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:42:50.589382 18723 external_mini_cluster.cc:1428] Setting key 8b76b8b33c2a6e9749385dbc0fc589fb
Aug 11 07:42:50 dist-test-slave-zkpd krb5kdc[5944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434170, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:50.598949 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.586999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:37553 (local address 127.18.72.254:39453)
0811 07:42:50.587127 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.587130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.587961 (+ 831us) server_negotiation.cc:407] Connection header received
0811 07:42:50.589486 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.589490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.589551 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.589643 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:50.591024 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.591796 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.593012 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.593226 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.593310 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.595586 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:50.595589 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:50.595591 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:50.595615 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:50.596868 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:50.597381 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:50.597384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:50.597385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:50.597459 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:50.597752 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:50.597754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:50.597755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:50.597921 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:50.597994 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.598634 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.598764 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:42:50.599989 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.587361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:39453 (local address 127.18.72.195:37553)
0811 07:42:50.587804 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.587847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:50.589245 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:50.589837 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:50.589850 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:50.590183 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:50.590892 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.590908 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.591953 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.591957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.592902 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.592913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.593352 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.593355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.593490 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.594035 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:50.594051 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:50.595442 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:50.596968 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:50.596973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:50.596984 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:50.597297 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:50.597538 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:50.597540 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:50.597541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:50.597680 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:50.598002 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:50.598007 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:50.598288 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 07:42:50.598564 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 07:42:50.598924 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":120,"threads_started":1}
I20260811 07:42:50.601274 6494 heartbeater.cc:347] Connected to a master server at 127.18.72.254:39453
I20260811 07:42:50.601541 6494 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:50.602101 6494 heartbeater.cc:511] Master 127.18.72.254:39453 requested a full tablet report, sending...
I20260811 07:42:50.603199 5997 ts_manager.cc:194] Registered new tserver with Master: a436a84c5e404443ae058bb3be0a00ee (127.18.72.195:35743)
I20260811 07:42:50.603790 5997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:37553
I20260811 07:42:50.614593 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:42:50.623661 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.616535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.18.72.254:39453)
0811 07:42:50.616682 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.616686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.616894 (+ 208us) server_negotiation.cc:407] Connection header received
0811 07:42:50.616986 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.616989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.617025 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.617082 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:50.617816 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.618289 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.619104 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.619307 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.619378 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.620867 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:50.620869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:50.620872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:50.620892 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:50.622042 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:50.622499 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:50.622502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:50.622503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:50.622571 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:50.622879 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:50.622881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:50.622882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:50.623056 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:50.623160 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.623392 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.623490 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:42:50.626395 5997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:42:50.629343 5997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:42:50.643107 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.637043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37701 (local address 127.0.0.1:54016)
0811 07:42:50.637705 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.637752 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:50.637871 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:50.638604 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:50.638612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:50.638717 (+ 105us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:50.638876 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.638886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.640265 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.640268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.641566 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.641576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.642593 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.642600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.642701 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.642736 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 07:42:50.642849 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 07:42:50.642961 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"spinlock_wait_cycles":4864,"thread_start_us":70,"threads_started":1}
I20260811 07:42:50.643150 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.637208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.18.72.193:37701)
0811 07:42:50.638002 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.638022 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.638045 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:42:50.638126 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.638132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.638335 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.638516 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:50.638996 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.640143 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.641690 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.642465 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.642578 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.642713 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.642901 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.643011 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":141,"threads_started":1}
I20260811 07:42:50.643416 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.638011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.18.72.194:44567)
0811 07:42:50.638480 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.638493 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.638673 (+ 180us) server_negotiation.cc:407] Connection header received
0811 07:42:50.639049 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.639054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.639181 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.639348 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:50.639577 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.640532 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.641861 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.642513 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.642632 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.642759 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.643181 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.643276 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":115,"threads_started":1}
I20260811 07:42:50.643500 6508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.637222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:44567 (local address 127.0.0.1:49024)
0811 07:42:50.638551 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.638563 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:50.638723 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:50.639336 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:50.639339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:50.639357 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:50.639461 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.639471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.640647 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.640652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.641736 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.641747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.642834 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.642840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.642929 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.642949 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 07:42:50.643012 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 07:42:50.643062 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"spinlock_wait_cycles":134144,"thread_start_us":104,"threads_started":1}
I20260811 07:42:50.643913 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.637535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:35743 (local address 127.0.0.1:43378)
0811 07:42:50.638856 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.638867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:50.638954 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:50.639510 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:50.639515 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:50.639536 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:50.639689 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.639697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.640972 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.640975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.641995 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.642008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.643340 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.643353 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.643457 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.643642 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 07:42:50.643710 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 07:42:50.643774 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"thread_start_us":101,"threads_started":1}
I20260811 07:42:50.644299 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.637827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.18.72.195:35743)
0811 07:42:50.638234 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.638244 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.638967 (+ 723us) server_negotiation.cc:407] Connection header received
0811 07:42:50.639045 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.639050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.639225 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.639385 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:50.639824 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.640821 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.642115 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.642954 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.643050 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.643598 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.644076 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.644170 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":128,"threads_started":1}
I20260811 07:42:50.646095 6134 tablet_service.cc:1467] Processing CreateTablet for tablet c8719f6203d941768ed5e052e469bdd9 (DEFAULT_TABLE table=test-table [id=e2688689924b462ba87ecf3b5d31b485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:50.646088 6281 tablet_service.cc:1467] Processing CreateTablet for tablet c8719f6203d941768ed5e052e469bdd9 (DEFAULT_TABLE table=test-table [id=e2688689924b462ba87ecf3b5d31b485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:50.647217 6281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8719f6203d941768ed5e052e469bdd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:50.647217 6134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8719f6203d941768ed5e052e469bdd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:50.648196 6424 tablet_service.cc:1467] Processing CreateTablet for tablet c8719f6203d941768ed5e052e469bdd9 (DEFAULT_TABLE table=test-table [id=e2688689924b462ba87ecf3b5d31b485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:50.649135 6424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8719f6203d941768ed5e052e469bdd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:50.653381 6513 tablet_bootstrap.cc:492] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574: Bootstrap starting.
I20260811 07:42:50.654798 6514 tablet_bootstrap.cc:492] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce: Bootstrap starting.
I20260811 07:42:50.655843 6513 tablet_bootstrap.cc:654] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:50.656302 6515 tablet_bootstrap.cc:492] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: Bootstrap starting.
I20260811 07:42:50.656780 6513 log.cc:824] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:50.656850 6514 tablet_bootstrap.cc:654] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:50.658043 6514 log.cc:824] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:50.658334 6515 tablet_bootstrap.cc:654] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:50.658999 6513 tablet_bootstrap.cc:492] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574: No bootstrap required, opened a new log
I20260811 07:42:50.659204 6513 ts_tablet_manager.cc:1397] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 07:42:50.659574 6515 log.cc:824] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:50.659799 6514 tablet_bootstrap.cc:492] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce: No bootstrap required, opened a new log
I20260811 07:42:50.660012 6514 ts_tablet_manager.cc:1397] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 07:42:50.661630 6515 tablet_bootstrap.cc:492] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: No bootstrap required, opened a new log
I20260811 07:42:50.661813 6515 ts_tablet_manager.cc:1397] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 07:42:50.663209 6514 raft_consensus.cc:348] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.663442 6514 raft_consensus.cc:374] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:50.663496 6514 raft_consensus.cc:729] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4daea56190354526af4ab10348c288ce, State: Initialized, Role: FOLLOWER
I20260811 07:42:50.663483 6513 raft_consensus.cc:348] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.663753 6513 raft_consensus.cc:374] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:50.663808 6513 raft_consensus.cc:729] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc7118365ec412e8a6118e6b3bd1574, State: Initialized, Role: FOLLOWER
I20260811 07:42:50.663980 6514 consensus_queue.cc:260] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.664249 6513 consensus_queue.cc:260] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.664695 6352 heartbeater.cc:503] Master 127.18.72.254:39453 was elected leader, sending a full tablet report...
I20260811 07:42:50.664951 6514 ts_tablet_manager.cc:1428] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:42:50.665481 6515 raft_consensus.cc:348] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.665721 6515 raft_consensus.cc:374] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:50.665778 6515 raft_consensus.cc:729] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a436a84c5e404443ae058bb3be0a00ee, State: Initialized, Role: FOLLOWER
I20260811 07:42:50.666168 6515 consensus_queue.cc:260] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.666368 6513 ts_tablet_manager.cc:1428] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 07:42:50.666721 6206 heartbeater.cc:503] Master 127.18.72.254:39453 was elected leader, sending a full tablet report...
I20260811 07:42:50.667075 6515 ts_tablet_manager.cc:1428] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 07:42:50.667102 6494 heartbeater.cc:503] Master 127.18.72.254:39453 was elected leader, sending a full tablet report...
I20260811 07:42:50.718806 6520 raft_consensus.cc:482] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:50.719012 6520 raft_consensus.cc:504] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.720156 6520 leader_election.cc:288] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4daea56190354526af4ab10348c288ce (127.18.72.194:44567), a436a84c5e404443ae058bb3be0a00ee (127.18.72.195:35743)
I20260811 07:42:50.724435 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.720518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:35743 (local address 127.18.72.193:60927)
0811 07:42:50.720910 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.720926 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:50.721066 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:50.721359 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:50.721362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:50.721400 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:50.721563 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.721573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.722301 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.722304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.723372 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.723383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.724070 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.724072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.724145 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.724169 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 07:42:50.724230 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 07:42:50.724284 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":16,"spinlock_wait_cycles":25088,"thread_start_us":102,"threads_started":1}
I20260811 07:42:50.724457 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.720615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:60927 (local address 127.18.72.195:35743)
0811 07:42:50.720756 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.720761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.721017 (+ 256us) server_negotiation.cc:407] Connection header received
0811 07:42:50.721184 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.721187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.721244 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.721335 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:50.721678 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.722193 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.723502 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.723965 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.724082 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.724137 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.724264 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.724330 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 07:42:50.725026 6444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8719f6203d941768ed5e052e469bdd9" candidate_uuid: "ebc7118365ec412e8a6118e6b3bd1574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a436a84c5e404443ae058bb3be0a00ee" is_pre_election: true
I20260811 07:42:50.725155 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.720518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:44567 (local address 127.18.72.193:59723)
0811 07:42:50.720910 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.720926 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:50.721066 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:50.721359 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:50.721362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:50.721986 (+ 624us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:50.722095 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.722102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.722842 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.722846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.723790 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:50.723799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.724734 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.724738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:50.724844 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.724864 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 07:42:50.724948 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 07:42:50.725019 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"spinlock_wait_cycles":9472,"thread_start_us":132,"threads_started":1}
I20260811 07:42:50.725172 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.720701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:59723 (local address 127.18.72.194:44567)
0811 07:42:50.720818 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.720821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.721017 (+ 196us) server_negotiation.cc:407] Connection header received
0811 07:42:50.721183 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.721187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.721241 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.721334 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:50.722231 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.722733 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.723937 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.724533 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.724656 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.724705 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.724981 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.725053 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:42:50.725369 6444 raft_consensus.cc:2454] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebc7118365ec412e8a6118e6b3bd1574 in term 0.
I20260811 07:42:50.725769 6304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8719f6203d941768ed5e052e469bdd9" candidate_uuid: "ebc7118365ec412e8a6118e6b3bd1574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4daea56190354526af4ab10348c288ce" is_pre_election: true
I20260811 07:42:50.725898 6083 leader_election.cc:302] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a436a84c5e404443ae058bb3be0a00ee, ebc7118365ec412e8a6118e6b3bd1574; no voters:
I20260811 07:42:50.726068 6304 raft_consensus.cc:2454] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebc7118365ec412e8a6118e6b3bd1574 in term 0.
I20260811 07:42:50.726233 6520 raft_consensus.cc:2781] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:42:50.726333 6520 raft_consensus.cc:482] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:42:50.726398 6520 raft_consensus.cc:3037] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:50.727484 6520 raft_consensus.cc:504] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.727856 6520 leader_election.cc:288] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [CANDIDATE]: Term 1 election: Requested vote from peers 4daea56190354526af4ab10348c288ce (127.18.72.194:44567), a436a84c5e404443ae058bb3be0a00ee (127.18.72.195:35743)
I20260811 07:42:50.728196 6444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8719f6203d941768ed5e052e469bdd9" candidate_uuid: "ebc7118365ec412e8a6118e6b3bd1574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a436a84c5e404443ae058bb3be0a00ee"
I20260811 07:42:50.728211 6304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8719f6203d941768ed5e052e469bdd9" candidate_uuid: "ebc7118365ec412e8a6118e6b3bd1574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4daea56190354526af4ab10348c288ce"
I20260811 07:42:50.728305 6444 raft_consensus.cc:3037] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:50.728338 6304 raft_consensus.cc:3037] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:50.729343 6444 raft_consensus.cc:2454] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebc7118365ec412e8a6118e6b3bd1574 in term 1.
I20260811 07:42:50.729426 6304 raft_consensus.cc:2454] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebc7118365ec412e8a6118e6b3bd1574 in term 1.
I20260811 07:42:50.729625 6083 leader_election.cc:302] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a436a84c5e404443ae058bb3be0a00ee, ebc7118365ec412e8a6118e6b3bd1574; no voters:
I20260811 07:42:50.729838 6520 raft_consensus.cc:2781] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:50.730065 6520 raft_consensus.cc:686] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 1 LEADER]: Becoming Leader. State: Replica: ebc7118365ec412e8a6118e6b3bd1574, State: Running, Role: LEADER
I20260811 07:42:50.730422 6520 consensus_queue.cc:237] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } }
I20260811 07:42:50.733743 5997 catalog_manager.cc:5495] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebc7118365ec412e8a6118e6b3bd1574 (127.18.72.193). New cstate: current_term: 1 leader_uuid: "ebc7118365ec412e8a6118e6b3bd1574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc7118365ec412e8a6118e6b3bd1574" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 37701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 } health_report { overall_health: UNKNOWN } } }
I20260811 07:42:50.766063 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.762798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.18.72.193:37701)
0811 07:42:50.762966 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:50.762970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:50.762985 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:42:50.763052 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:50.763054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:50.763105 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:50.763197 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:42:50.763420 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.763951 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.764761 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:50.764969 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:50.765048 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:50.765336 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:42:50.765716 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:42:50.765827 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:50.765927 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 07:42:50.765959 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 07:42:50.771751 6304 raft_consensus.cc:1261] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 1 FOLLOWER]: Refusing update from remote peer ebc7118365ec412e8a6118e6b3bd1574: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:50.771751 6444 raft_consensus.cc:1261] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 1 FOLLOWER]: Refusing update from remote peer ebc7118365ec412e8a6118e6b3bd1574: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:50.772476 6520 consensus_queue.cc:1035] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a436a84c5e404443ae058bb3be0a00ee" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 35743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:50.772677 6525 consensus_queue.cc:1035] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4daea56190354526af4ab10348c288ce" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 44567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:50.783098 6531 mvcc.cc:205] Tried to move back new op lower bound from 7317234363474210816 to 7317234363314876416. Current Snapshot: MvccSnapshot[applied={T|T < 7317234363474210816 or (T in {7317234363474210816})}]
I20260811 07:42:50.787773 5991 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:50.787971 5991 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:50.790362 5991 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86463df27d0e40b6abdab80c73c12a7b: Sending DeleteTablet for 3 replicas of tablet c8719f6203d941768ed5e052e469bdd9
I20260811 07:42:50.791157 6281 tablet_service.cc:1514] Processing DeleteTablet for tablet c8719f6203d941768ed5e052e469bdd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:50 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:49024
I20260811 07:42:50.791216 6424 tablet_service.cc:1514] Processing DeleteTablet for tablet c8719f6203d941768ed5e052e469bdd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:50 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43378
I20260811 07:42:50.791318 6134 tablet_service.cc:1514] Processing DeleteTablet for tablet c8719f6203d941768ed5e052e469bdd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:50 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:54016
I20260811 07:42:50.791666 6540 tablet_replica.cc:330] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: stopping tablet replica
I20260811 07:42:50.791709 6541 tablet_replica.cc:330] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574: stopping tablet replica
I20260811 07:42:50.792023 6541 raft_consensus.cc:2229] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:42:50.792033 6540 raft_consensus.cc:2229] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:50.792337 6539 tablet_replica.cc:330] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce: stopping tablet replica
I20260811 07:42:50.792339 6540 raft_consensus.cc:2258] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:50.792339 6541 raft_consensus.cc:2258] T c8719f6203d941768ed5e052e469bdd9 P ebc7118365ec412e8a6118e6b3bd1574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:50.792668 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 6062
I20260811 07:42:50.795061 6540 ts_tablet_manager.cc:1905] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:50.798571 6540 ts_tablet_manager.cc:1918] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:50.798645 6540 log.cc:1196] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/c8719f6203d941768ed5e052e469bdd9
I20260811 07:42:50.798903 6540 ts_tablet_manager.cc:1939] T c8719f6203d941768ed5e052e469bdd9 P a436a84c5e404443ae058bb3be0a00ee: Deleting consensus metadata
I20260811 07:42:50.799927 5982 catalog_manager.cc:4841] TS a436a84c5e404443ae058bb3be0a00ee (127.18.72.195:35743): tablet c8719f6203d941768ed5e052e469bdd9 (table test-table [id=e2688689924b462ba87ecf3b5d31b485]) successfully deleted
I20260811 07:42:50.800608 6539 raft_consensus.cc:2229] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:50.800904 6539 raft_consensus.cc:2258] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:50.802071 6539 ts_tablet_manager.cc:1905] T c8719f6203d941768ed5e052e469bdd9 P 4daea56190354526af4ab10348c288ce: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 07:42:50.804415 5978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:37701 (error 108)
I20260811 07:42:50.804666 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 6211
I20260811 07:42:50.806260 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.805981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:37701 (local address 127.0.0.1:54040)
0811 07:42:50.806107 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.806182 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 07:42:50.806468 5978 catalog_manager.cc:4584] TS ebc7118365ec412e8a6118e6b3bd1574 (127.18.72.193:37701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8719f6203d941768ed5e052e469bdd9: Network error: Client connection negotiation failed: client connection to 127.18.72.193:37701: connect: Connection refused (error 111)
I20260811 07:42:50.811893 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 6357
I20260811 07:42:50.815083 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:50.812022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:44567 (local address 127.0.0.1:49040)
0811 07:42:50.814942 (+ 2920us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:50.815019 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.194:44567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2838}
I20260811 07:42:50.819681 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 5959
2026-08-11T07:42:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2359 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T07:42:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:50Z Disabled control of system clock
I20260811 07:42:50.852366 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:34463
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:38023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:34463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:42:50.963668 6547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:50.963958 6547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:50.964003 6547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:50.967902 6547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:50.967950 6547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:50.967967 6547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:50.967983 6547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:50.967998 6547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:50.972587 6547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:34463
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:34463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:50.973531 6547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:50.974810 6547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:50.983256 6553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:50.983376 6555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:50.983546 6552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:50.983449 6547 server_base.cc:1034] running on GCE node
I20260811 07:42:50.984193 6547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:50.985304 6547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:50.986475 6547 hybrid_clock.cc:648] HybridClock initialized: now 1786434170986444 us; error 40 us; skew 500 ppm
I20260811 07:42:50.988526 6547 webserver.cc:460] Webserver started at http://127.18.72.254:35567/ using document root <none> and password file <none>
I20260811 07:42:50.989125 6547 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:50.989187 6547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:50.989557 6547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:50.991647 6547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "c032a5863ca54bbe82ee2727ac91fbe9"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "da985d623861d3465ffafeb2b489606d"
server_key_iv: "928fe5a7279c7ca15738f6791c80096a"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.992261 6547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "c032a5863ca54bbe82ee2727ac91fbe9"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "da985d623861d3465ffafeb2b489606d"
server_key_iv: "928fe5a7279c7ca15738f6791c80096a"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.995978 6547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:42:50.998410 6561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:50.999584 6547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:50.999748 6547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "c032a5863ca54bbe82ee2727ac91fbe9"
format_stamp: "Formatted at 2026-08-11 07:42:50 on dist-test-slave-zkpd"
server_key: "da985d623861d3465ffafeb2b489606d"
server_key_iv: "928fe5a7279c7ca15738f6791c80096a"
server_key_version: "encryptionkey@0"
I20260811 07:42:50.999857 6547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:51.022090 6547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:51.022765 6547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:51.022960 6547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:51.030071 6547 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:34463
I20260811 07:42:51.030079 6618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:34463 every 8 connection(s)
I20260811 07:42:51.030977 6547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:51.034057 6619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:51.038160 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 6547
I20260811 07:42:51.038267 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:42:51.038569 18723 external_mini_cluster.cc:1428] Setting key f0b27748124bf96c75d0d4989ea34a47
I20260811 07:42:51.039825 6619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9: Bootstrap starting.
I20260811 07:42:51.042539 6619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:51.043428 6619 log.cc:824] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:51.045529 6619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9: No bootstrap required, opened a new log
I20260811 07:42:51.049407 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.040053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.18.72.254:34463)
0811 07:42:51.040705 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.040725 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.040766 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:42:51.041614 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.041637 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.042130 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.042488 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:51.043030 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.044483 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.045496 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.046172 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.046325 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.047182 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:51.047189 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:51.047205 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:51.047705 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:51.047773 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:51.047798 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:51.047932 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.048074 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.048342 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":103,"threads_started":1}
I20260811 07:42:51.050226 6619 raft_consensus.cc:348] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c032a5863ca54bbe82ee2727ac91fbe9" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34463 } }
I20260811 07:42:51.050500 6619 raft_consensus.cc:374] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:51.050557 6619 raft_consensus.cc:729] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c032a5863ca54bbe82ee2727ac91fbe9, State: Initialized, Role: FOLLOWER
I20260811 07:42:51.051013 6619 consensus_queue.cc:260] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c032a5863ca54bbe82ee2727ac91fbe9" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34463 } }
I20260811 07:42:51.051134 6619 raft_consensus.cc:388] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:51.051216 6619 raft_consensus.cc:482] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:51.051332 6619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:51.052351 6619 raft_consensus.cc:504] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c032a5863ca54bbe82ee2727ac91fbe9" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34463 } }
I20260811 07:42:51.052690 6619 leader_election.cc:302] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c032a5863ca54bbe82ee2727ac91fbe9; no voters:
I20260811 07:42:51.052977 6619 leader_election.cc:288] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:42:51.053093 6624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:51.053310 6624 raft_consensus.cc:686] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [term 1 LEADER]: Becoming Leader. State: Replica: c032a5863ca54bbe82ee2727ac91fbe9, State: Running, Role: LEADER
I20260811 07:42:51.053627 6624 consensus_queue.cc:237] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c032a5863ca54bbe82ee2727ac91fbe9" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34463 } }
I20260811 07:42:51.054219 6619 sys_catalog.cc:564] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:51.055254 6625 sys_catalog.cc:455] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c032a5863ca54bbe82ee2727ac91fbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c032a5863ca54bbe82ee2727ac91fbe9" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34463 } } }
I20260811 07:42:51.055393 6625 sys_catalog.cc:458] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:51.055649 6626 sys_catalog.cc:455] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c032a5863ca54bbe82ee2727ac91fbe9. Latest consensus state: current_term: 1 leader_uuid: "c032a5863ca54bbe82ee2727ac91fbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c032a5863ca54bbe82ee2727ac91fbe9" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 34463 } } }
I20260811 07:42:51.055765 6626 sys_catalog.cc:458] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:51.056069 6633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:51.059187 6633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:51.064972 6633 catalog_manager.cc:1347] Generated new cluster ID: 5b67dfb54c6d4e77a0d496c9491a0453
I20260811 07:42:51.065040 6633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:51.087867 6633 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:42:51.088737 6633 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:51.092866 6633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9: Generated new TSK 0
I20260811 07:42:51.093469 6633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:42:51.102927 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34463
--builtin_ntp_servers=127.18.72.212:38023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:42:51.234364 6643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:51.234596 6643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:51.234652 6643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:51.238533 6643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:51.238575 6643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:51.238667 6643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:42:51.243623 6643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:51.244764 6643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:51.246238 6643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:51.255780 6650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:51.255833 6653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:51.255793 6651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:51.255846 6643 server_base.cc:1034] running on GCE node
I20260811 07:42:51.256475 6643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:51.257050 6643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:51.258203 6643 hybrid_clock.cc:648] HybridClock initialized: now 1786434171258169 us; error 44 us; skew 500 ppm
I20260811 07:42:51.260003 6643 webserver.cc:460] Webserver started at http://127.18.72.193:45347/ using document root <none> and password file <none>
I20260811 07:42:51.260558 6643 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:51.260618 6643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:51.260861 6643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:51.262815 6643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "7854366b38174865b9fa64688c58823d"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "1c8e6bfe7c9381d07f247f1d289b651a"
server_key_iv: "06db7ad50d91f2552c5df4028f56d402"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.263357 6643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "7854366b38174865b9fa64688c58823d"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "1c8e6bfe7c9381d07f247f1d289b651a"
server_key_iv: "06db7ad50d91f2552c5df4028f56d402"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.266880 6643 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 07:42:51.269219 6659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.270355 6643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:42:51.270483 6643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "7854366b38174865b9fa64688c58823d"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "1c8e6bfe7c9381d07f247f1d289b651a"
server_key_iv: "06db7ad50d91f2552c5df4028f56d402"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.270586 6643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:51.282426 6643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:51.283033 6643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:51.283215 6643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:51.283761 6643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:51.284700 6643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:51.284778 6643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.284871 6643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:51.284919 6643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.293828 6643 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:44573
I20260811 07:42:51.293856 6779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:44573 every 8 connection(s)
I20260811 07:42:51.294689 6643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:42:51.300938 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 6643
I20260811 07:42:51.301064 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:42:51.301383 18723 external_mini_cluster.cc:1428] Setting key 36a441d456b9abfa550e553702b14f30
I20260811 07:42:51.303499 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34463
--builtin_ntp_servers=127.18.72.212:38023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:42:51.304659 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.296536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:57405 (local address 127.18.72.254:34463)
0811 07:42:51.296668 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.296672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.297751 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 07:42:51.299225 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.299229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.299292 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.299389 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:51.300389 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.301138 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.302423 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.302723 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.302844 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.304168 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:51.304171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:51.304173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:51.304185 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:51.304202 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:51.304210 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:51.304282 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.304453 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.304513 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:42:51.305723 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.296946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:34463 (local address 127.18.72.193:57405)
0811 07:42:51.297610 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.297651 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:51.298998 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:51.299575 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:51.299582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:51.299671 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:51.300246 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.300260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.301271 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.301274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.302307 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.302319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.302898 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.302902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.303065 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.303854 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:51.303856 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:51.303945 (+ 89us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:51.304020 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:51.304307 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:51.304316 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:51.304322 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:42:51.304453 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 07:42:51.304748 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260811 07:42:51.306986 6780 heartbeater.cc:347] Connected to a master server at 127.18.72.254:34463
I20260811 07:42:51.307236 6780 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:51.307749 6780 heartbeater.cc:511] Master 127.18.72.254:34463 requested a full tablet report, sending...
I20260811 07:42:51.309254 6581 ts_manager.cc:194] Registered new tserver with Master: 7854366b38174865b9fa64688c58823d (127.18.72.193:44573)
I20260811 07:42:51.310396 6581 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.193:57405
W20260811 07:42:51.420881 6783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:51.421132 6783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:51.421196 6783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:51.425046 6783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:51.425117 6783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:51.425197 6783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:42:51.429812 6783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:51.430873 6783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:51.432250 6783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:51.441581 6788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:51.441587 6791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:51.441618 6789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:51.441982 6783 server_base.cc:1034] running on GCE node
I20260811 07:42:51.442360 6783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:51.443006 6783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:51.444188 6783 hybrid_clock.cc:648] HybridClock initialized: now 1786434171444175 us; error 26 us; skew 500 ppm
I20260811 07:42:51.445971 6783 webserver.cc:460] Webserver started at http://127.18.72.194:46755/ using document root <none> and password file <none>
I20260811 07:42:51.446527 6783 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:51.446609 6783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:51.446854 6783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:51.448863 6783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "4e0d0ea08e3441bfad6720a6b3a9560c"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "8a858be3136791dc4dab3fdde42acc37"
server_key_iv: "be7117fb2a896a2e8e0a87e1aa2bc575"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.449402 6783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "4e0d0ea08e3441bfad6720a6b3a9560c"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "8a858be3136791dc4dab3fdde42acc37"
server_key_iv: "be7117fb2a896a2e8e0a87e1aa2bc575"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.453112 6783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:42:51.455521 6798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.456820 6783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:51.456940 6783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "4e0d0ea08e3441bfad6720a6b3a9560c"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "8a858be3136791dc4dab3fdde42acc37"
server_key_iv: "be7117fb2a896a2e8e0a87e1aa2bc575"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.457054 6783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:51.493745 6783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:51.494468 6783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:51.494675 6783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:51.495282 6783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:51.496308 6783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:51.496383 6783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.496471 6783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:51.496518 6783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.505432 6783 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:41599
I20260811 07:42:51.505467 6922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:41599 every 8 connection(s)
I20260811 07:42:51.506313 6783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:42:51.511505 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 6783
I20260811 07:42:51.511647 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:42:51.511979 18723 external_mini_cluster.cc:1428] Setting key a0afa1c9394dbbf6678115f7ce00e61d
I20260811 07:42:51.513803 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34463
--builtin_ntp_servers=127.18.72.212:38023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:42:51.515448 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.508105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:38317 (local address 127.18.72.254:34463)
0811 07:42:51.508237 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.508241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.509320 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 07:42:51.510724 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.510728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.510794 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.510892 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:51.511818 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.512442 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.513540 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.513768 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.513897 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.514970 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:51.514974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:51.514976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:51.514988 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:51.515013 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:51.515021 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:51.515092 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.515253 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.515311 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:42:51.516275 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.508531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:34463 (local address 127.18.72.194:38317)
0811 07:42:51.509180 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.509217 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:51.510510 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:51.511067 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:51.511074 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:51.511153 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:51.511655 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.511681 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.512564 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.512568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.513416 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.513426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.513943 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.513948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.514123 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.514726 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:51.514728 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:51.514765 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:51.514828 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:51.515107 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:51.515121 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:51.515126 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:42:51.515238 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 07:42:51.515519 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":125,"threads_started":1}
I20260811 07:42:51.517313 6923 heartbeater.cc:347] Connected to a master server at 127.18.72.254:34463
I20260811 07:42:51.517540 6923 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:51.518116 6923 heartbeater.cc:511] Master 127.18.72.254:34463 requested a full tablet report, sending...
I20260811 07:42:51.519287 6581 ts_manager.cc:194] Registered new tserver with Master: 4e0d0ea08e3441bfad6720a6b3a9560c (127.18.72.194:41599)
I20260811 07:42:51.519956 6581 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.194:38317
W20260811 07:42:51.628504 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:51.628749 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:51.628813 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:51.633311 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:51.633358 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:51.633450 6926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:42:51.638147 6926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:34463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:51.639050 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:51.640373 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:51.649837 6931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:51.649850 6935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:51.649850 6932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:51.650275 6926 server_base.cc:1034] running on GCE node
I20260811 07:42:51.650694 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:51.651324 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:51.652522 6926 hybrid_clock.cc:648] HybridClock initialized: now 1786434171652508 us; error 24 us; skew 500 ppm
I20260811 07:42:51.654361 6926 webserver.cc:460] Webserver started at http://127.18.72.195:36695/ using document root <none> and password file <none>
I20260811 07:42:51.654986 6926 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:51.655071 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:51.655278 6926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:51.657306 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "8135edd9ad5e4e2faaf1fdb55868daa6"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "7b45bd3f50e861f837a1fcadaf2348b2"
server_key_iv: "ed5997d808ba9409e12c1227c53355f0"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.657836 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "8135edd9ad5e4e2faaf1fdb55868daa6"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "7b45bd3f50e861f837a1fcadaf2348b2"
server_key_iv: "ed5997d808ba9409e12c1227c53355f0"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.661509 6926 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:42:51.663822 6941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.665002 6926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:51.665141 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "8135edd9ad5e4e2faaf1fdb55868daa6"
format_stamp: "Formatted at 2026-08-11 07:42:51 on dist-test-slave-zkpd"
server_key: "7b45bd3f50e861f837a1fcadaf2348b2"
server_key_iv: "ed5997d808ba9409e12c1227c53355f0"
server_key_version: "encryptionkey@0"
I20260811 07:42:51.665246 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:51.677943 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:51.678544 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:51.678720 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:51.679250 6926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:51.680228 6926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:51.680294 6926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.680364 6926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:51.680411 6926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:51.689388 6926 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:38097
I20260811 07:42:51.689422 7059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:38097 every 8 connection(s)
I20260811 07:42:51.690204 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:42:51.691488 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 6926
I20260811 07:42:51.691577 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:42:51.691887 18723 external_mini_cluster.cc:1428] Setting key 516f97157ac24bd21d8bd68785096298
I20260811 07:42:51.699424 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.692016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:35855 (local address 127.18.72.254:34463)
0811 07:42:51.692157 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.692162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.693444 (+ 1282us) server_negotiation.cc:407] Connection header received
0811 07:42:51.694962 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.694965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.695019 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.695091 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:51.696001 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.696530 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.697637 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.697873 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.697982 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.698952 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:51.698956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:51.698958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:51.698976 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:51.698995 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:51.699003 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:51.699077 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.699225 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.699286 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:42:51.700250 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.692494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:34463 (local address 127.18.72.195:35855)
0811 07:42:51.693293 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.693353 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:51.694766 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:51.695284 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:51.695291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:51.695383 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:51.695828 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.695840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.696654 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.696658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.697514 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.697523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.698019 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.698023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.698180 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.698711 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:51.698713 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:51.698750 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:51.698808 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:51.699080 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:51.699093 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:51.699099 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:42:51.699224 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 07:42:51.699503 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":141,"threads_started":1}
I20260811 07:42:51.701252 7060 heartbeater.cc:347] Connected to a master server at 127.18.72.254:34463
I20260811 07:42:51.701481 7060 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:51.701956 7060 heartbeater.cc:511] Master 127.18.72.254:34463 requested a full tablet report, sending...
I20260811 07:42:51.702929 6581 ts_manager.cc:194] Registered new tserver with Master: 8135edd9ad5e4e2faaf1fdb55868daa6 (127.18.72.195:38097)
I20260811 07:42:51.703467 6581 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.195:35855
I20260811 07:42:51.705523 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:42:51.711086 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.707330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.18.72.254:34463)
0811 07:42:51.707453 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.707457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.707630 (+ 173us) server_negotiation.cc:407] Connection header received
0811 07:42:51.707770 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.707773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.707814 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.707902 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:51.708194 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.708675 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.709509 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.709730 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.709832 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.710596 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:51.710600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:51.710603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:51.710615 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:51.710639 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:51.710647 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:51.710771 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.710916 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.710954 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:42:51.713485 6581 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:42:51.716464 6581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:42:51.730265 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.725005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:38097 (local address 127.0.0.1:35812)
0811 07:42:51.725682 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.725694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:51.725772 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:51.726287 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:51.726290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:51.726313 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:51.726411 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.726417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.727397 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.727401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.728667 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.728679 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.729685 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.729688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.729803 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.729841 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 07:42:51.729956 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 07:42:51.730121 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":74,"thread_start_us":87,"threads_started":1}
I20260811 07:42:51.730253 7079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.725391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.18.72.195:38097)
0811 07:42:51.725738 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.725751 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.725775 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:42:51.725881 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.725892 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.726054 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.726202 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:51.726532 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.727271 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.728793 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.729570 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.729685 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.729875 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.729986 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.730089 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":133,"threads_started":1}
I20260811 07:42:51.730283 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.724317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:41599 (local address 127.0.0.1:56612)
0811 07:42:51.724652 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.724701 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:51.724837 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:51.726410 (+ 1573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:51.726411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:51.726426 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:51.726512 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.726517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.727872 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.727876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.728914 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.728922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.729772 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.729776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.729878 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.729904 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:42:51.729954 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 07:42:51.730111 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":103,"thread_start_us":69,"threads_started":1}
I20260811 07:42:51.730729 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.724722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:44573 (local address 127.0.0.1:33128)
0811 07:42:51.725401 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.725415 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:51.725502 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:51.726121 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:51.726130 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:51.726158 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:51.726294 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.726305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.727753 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.727757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.729319 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.729331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.730330 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.730334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.730459 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.730494 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 07:42:51.730554 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 07:42:51.730613 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":405,"thread_start_us":119,"threads_started":1}
I20260811 07:42:51.731446 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.725165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33128 (local address 127.18.72.193:44573)
0811 07:42:51.725674 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.725687 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.725711 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:42:51.725805 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.725810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.725953 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.726114 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:51.726425 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.727615 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.729456 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.730193 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.730977 (+ 784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.731140 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.731191 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.731277 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":115,"threads_started":1}
I20260811 07:42:51.731442 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.724503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.18.72.194:41599)
0811 07:42:51.725199 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.725213 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.725239 (+ 26us) server_negotiation.cc:407] Connection header received
0811 07:42:51.725714 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.725720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.725945 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.726114 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:51.726631 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.727742 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.729029 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.729637 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.730957 (+ 1320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.731140 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.731191 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.731277 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":109,"threads_started":1}
I20260811 07:42:51.733494 6990 tablet_service.cc:1467] Processing CreateTablet for tablet 08dd0096318e4215b3e4385bd2542fff (DEFAULT_TABLE table=test-table [id=76d8a8d4860d4599aedd3aead44a940f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:51.733749 6850 tablet_service.cc:1467] Processing CreateTablet for tablet 08dd0096318e4215b3e4385bd2542fff (DEFAULT_TABLE table=test-table [id=76d8a8d4860d4599aedd3aead44a940f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:51.734642 6850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08dd0096318e4215b3e4385bd2542fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:51.734637 6990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08dd0096318e4215b3e4385bd2542fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:51.734277 6710 tablet_service.cc:1467] Processing CreateTablet for tablet 08dd0096318e4215b3e4385bd2542fff (DEFAULT_TABLE table=test-table [id=76d8a8d4860d4599aedd3aead44a940f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:51.735188 6710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08dd0096318e4215b3e4385bd2542fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:51.741345 7080 tablet_bootstrap.cc:492] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Bootstrap starting.
I20260811 07:42:51.742317 7082 tablet_bootstrap.cc:492] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d: Bootstrap starting.
I20260811 07:42:51.743371 7080 tablet_bootstrap.cc:654] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:51.744143 7080 log.cc:824] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:51.744212 7082 tablet_bootstrap.cc:654] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:51.744228 7081 tablet_bootstrap.cc:492] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: Bootstrap starting.
I20260811 07:42:51.745062 7082 log.cc:824] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:51.746078 7080 tablet_bootstrap.cc:492] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: No bootstrap required, opened a new log
I20260811 07:42:51.746227 7081 tablet_bootstrap.cc:654] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:51.746273 7080 ts_tablet_manager.cc:1397] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 07:42:51.747035 7082 tablet_bootstrap.cc:492] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d: No bootstrap required, opened a new log
I20260811 07:42:51.747073 7081 log.cc:824] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:51.747223 7082 ts_tablet_manager.cc:1397] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 07:42:51.748991 7081 tablet_bootstrap.cc:492] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: No bootstrap required, opened a new log
I20260811 07:42:51.749155 7081 ts_tablet_manager.cc:1397] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 07:42:51.750058 7080 raft_consensus.cc:348] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.750280 7080 raft_consensus.cc:374] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:51.750334 7080 raft_consensus.cc:729] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0d0ea08e3441bfad6720a6b3a9560c, State: Initialized, Role: FOLLOWER
I20260811 07:42:51.750753 7080 consensus_queue.cc:260] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.751391 6923 heartbeater.cc:503] Master 127.18.72.254:34463 was elected leader, sending a full tablet report...
I20260811 07:42:51.751446 7082 raft_consensus.cc:348] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.751681 7082 raft_consensus.cc:374] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:51.751739 7082 raft_consensus.cc:729] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7854366b38174865b9fa64688c58823d, State: Initialized, Role: FOLLOWER
I20260811 07:42:51.752206 7082 consensus_queue.cc:260] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.752755 7080 ts_tablet_manager.cc:1428] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:42:51.753109 7082 ts_tablet_manager.cc:1428] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 07:42:51.753441 6780 heartbeater.cc:503] Master 127.18.72.254:34463 was elected leader, sending a full tablet report...
I20260811 07:42:51.753302 7081 raft_consensus.cc:348] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.753530 7081 raft_consensus.cc:374] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:51.753585 7081 raft_consensus.cc:729] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8135edd9ad5e4e2faaf1fdb55868daa6, State: Initialized, Role: FOLLOWER
I20260811 07:42:51.754062 7081 consensus_queue.cc:260] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.754825 7060 heartbeater.cc:503] Master 127.18.72.254:34463 was elected leader, sending a full tablet report...
I20260811 07:42:51.755683 7081 ts_tablet_manager.cc:1428] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
W20260811 07:42:51.757189 6924 tablet.cc:2369] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:42:51.775662 7086 raft_consensus.cc:482] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:51.775818 7086 raft_consensus.cc:504] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.776824 7086 leader_election.cc:288] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8135edd9ad5e4e2faaf1fdb55868daa6 (127.18.72.195:38097), 7854366b38174865b9fa64688c58823d (127.18.72.193:44573)
I20260811 07:42:51.780457 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.777021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:44573 (local address 127.18.72.194:36243)
0811 07:42:51.777208 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.777223 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:51.777319 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:51.777619 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:51.777622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:51.777640 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:51.777775 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.777783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.778549 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.778552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.779367 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.779377 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.780038 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.780044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.780150 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.780176 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 07:42:51.780244 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 07:42:51.780298 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":4}
I20260811 07:42:51.780483 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.777091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:36243 (local address 127.18.72.193:44573)
0811 07:42:51.777199 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.777204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.777273 (+ 69us) server_negotiation.cc:407] Connection header received
0811 07:42:51.777429 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.777433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.777496 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.777568 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:51.777885 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.778444 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.779496 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.779882 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.780022 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.780094 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.780281 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.780362 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:42:51.780556 7079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.777091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:46757 (local address 127.18.72.195:38097)
0811 07:42:51.777199 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.777204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.777518 (+ 314us) server_negotiation.cc:407] Connection header received
0811 07:42:51.777636 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.777639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.777684 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.777759 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:51.778056 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.778544 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.779533 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.779964 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.780083 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.780133 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.780366 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.780439 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:42:51.780850 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.777021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:38097 (local address 127.18.72.194:46757)
0811 07:42:51.777416 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.777431 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:51.777536 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:51.777777 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:51.777780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:51.777797 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:51.777948 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.777954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.778651 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.778653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.779396 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:51.779404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.780116 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.780121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:51.780209 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.780247 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 07:42:51.780333 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 07:42:51.780391 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":30,"thread_start_us":105,"threads_started":1}
I20260811 07:42:51.781065 6732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08dd0096318e4215b3e4385bd2542fff" candidate_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7854366b38174865b9fa64688c58823d" is_pre_election: true
I20260811 07:42:51.781340 6732 raft_consensus.cc:2454] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0d0ea08e3441bfad6720a6b3a9560c in term 0.
I20260811 07:42:51.781384 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08dd0096318e4215b3e4385bd2542fff" candidate_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" is_pre_election: true
I20260811 07:42:51.781620 7011 raft_consensus.cc:2454] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0d0ea08e3441bfad6720a6b3a9560c in term 0.
I20260811 07:42:51.781816 6802 leader_election.cc:302] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0d0ea08e3441bfad6720a6b3a9560c, 7854366b38174865b9fa64688c58823d; no voters:
I20260811 07:42:51.782097 7086 raft_consensus.cc:2781] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:42:51.782171 7086 raft_consensus.cc:482] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:42:51.782218 7086 raft_consensus.cc:3037] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:51.783162 7086 raft_consensus.cc:504] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.783511 7086 leader_election.cc:288] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [CANDIDATE]: Term 1 election: Requested vote from peers 8135edd9ad5e4e2faaf1fdb55868daa6 (127.18.72.195:38097), 7854366b38174865b9fa64688c58823d (127.18.72.193:44573)
I20260811 07:42:51.783794 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08dd0096318e4215b3e4385bd2542fff" candidate_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6"
I20260811 07:42:51.783851 6732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08dd0096318e4215b3e4385bd2542fff" candidate_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7854366b38174865b9fa64688c58823d"
I20260811 07:42:51.783946 7011 raft_consensus.cc:3037] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:51.783969 6732 raft_consensus.cc:3037] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:51.785212 6732 raft_consensus.cc:2454] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0d0ea08e3441bfad6720a6b3a9560c in term 1.
I20260811 07:42:51.785212 7011 raft_consensus.cc:2454] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0d0ea08e3441bfad6720a6b3a9560c in term 1.
I20260811 07:42:51.785524 6801 leader_election.cc:302] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0d0ea08e3441bfad6720a6b3a9560c, 8135edd9ad5e4e2faaf1fdb55868daa6; no voters:
I20260811 07:42:51.785676 7086 raft_consensus.cc:2781] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:51.785864 7086 raft_consensus.cc:686] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 1 LEADER]: Becoming Leader. State: Replica: 4e0d0ea08e3441bfad6720a6b3a9560c, State: Running, Role: LEADER
I20260811 07:42:51.786162 7086 consensus_queue.cc:237] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } }
I20260811 07:42:51.788991 6580 catalog_manager.cc:5495] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0d0ea08e3441bfad6720a6b3a9560c (127.18.72.194). New cstate: current_term: 1 leader_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e0d0ea08e3441bfad6720a6b3a9560c" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 41599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 } health_report { overall_health: UNKNOWN } } }
W20260811 07:42:51.795759 6781 tablet.cc:2369] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:42:51.808322 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.804977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56618 (local address 127.18.72.194:41599)
0811 07:42:51.805087 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:51.805091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:51.805108 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:42:51.805241 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:51.805243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:51.805295 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:51.805383 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:42:51.805577 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.806110 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.806939 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:51.807144 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:51.807217 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:51.807473 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:42:51.807851 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:42:51.808017 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:51.808100 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 07:42:51.808146 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 07:42:51.813911 6732 raft_consensus.cc:1261] T 08dd0096318e4215b3e4385bd2542fff P 7854366b38174865b9fa64688c58823d [term 1 FOLLOWER]: Refusing update from remote peer 4e0d0ea08e3441bfad6720a6b3a9560c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:51.813912 7011 raft_consensus.cc:1261] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0d0ea08e3441bfad6720a6b3a9560c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:51.814615 7086 consensus_queue.cc:1035] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7854366b38174865b9fa64688c58823d" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:51.814805 7090 consensus_queue.cc:1035] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8135edd9ad5e4e2faaf1fdb55868daa6" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 38097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:51.823251 7093 mvcc.cc:205] Tried to move back new op lower bound from 7317234367743029248 to 7317234367638544384. Current Snapshot: MvccSnapshot[applied={T|T < 7317234367743029248 or (T in {7317234367743029248})}]
I20260811 07:42:51.823874 7094 mvcc.cc:205] Tried to move back new op lower bound from 7317234367743029248 to 7317234367638544384. Current Snapshot: MvccSnapshot[applied={T|T < 7317234367743029248}]
I20260811 07:42:51.829521 6580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:51.829694 6580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:51.832463 6580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c032a5863ca54bbe82ee2727ac91fbe9: Sending DeleteTablet for 3 replicas of tablet 08dd0096318e4215b3e4385bd2542fff
I20260811 07:42:51.833341 6710 tablet_service.cc:1514] Processing DeleteTablet for tablet 08dd0096318e4215b3e4385bd2542fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:51 UTC) from {username='slave'} at 127.0.0.1:33128
I20260811 07:42:51.833377 6990 tablet_service.cc:1514] Processing DeleteTablet for tablet 08dd0096318e4215b3e4385bd2542fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:51 UTC) from {username='slave'} at 127.0.0.1:35812
I20260811 07:42:51.833956 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 6643
I20260811 07:42:51.834007 7102 tablet_replica.cc:330] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: stopping tablet replica
I20260811 07:42:51.834059 6850 tablet_service.cc:1514] Processing DeleteTablet for tablet 08dd0096318e4215b3e4385bd2542fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:51 UTC) from {username='slave'} at 127.0.0.1:56612
I20260811 07:42:51.835089 7102 raft_consensus.cc:2229] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:51.835439 7102 raft_consensus.cc:2258] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:51.837937 7104 tablet_replica.cc:330] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: stopping tablet replica
I20260811 07:42:51.838297 7104 raft_consensus.cc:2229] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:42:51.838672 7104 raft_consensus.cc:2258] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:51.840106 7104 ts_tablet_manager.cc:1905] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:51.842267 7104 ts_tablet_manager.cc:1918] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:51.842386 7104 log.cc:1196] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/08dd0096318e4215b3e4385bd2542fff
I20260811 07:42:51.842728 7104 ts_tablet_manager.cc:1939] T 08dd0096318e4215b3e4385bd2542fff P 4e0d0ea08e3441bfad6720a6b3a9560c: Deleting consensus metadata
I20260811 07:42:51.844089 7102 ts_tablet_manager.cc:1905] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:51.844193 6564 catalog_manager.cc:4841] TS 4e0d0ea08e3441bfad6720a6b3a9560c (127.18.72.194:41599): tablet 08dd0096318e4215b3e4385bd2542fff (table test-table [id=76d8a8d4860d4599aedd3aead44a940f]) successfully deleted
W20260811 07:42:51.845916 6566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:44573 (error 108)
I20260811 07:42:51.846769 7102 ts_tablet_manager.cc:1918] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:51.847035 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:51.846670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:44573 (local address 127.0.0.1:33132)
0811 07:42:51.846822 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:51.846901 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:44573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 07:42:51.847052 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 6783
I20260811 07:42:51.847429 7102 log.cc:1196] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/08dd0096318e4215b3e4385bd2542fff
I20260811 07:42:51.848070 7102 ts_tablet_manager.cc:1939] T 08dd0096318e4215b3e4385bd2542fff P 8135edd9ad5e4e2faaf1fdb55868daa6: Deleting consensus metadata
W20260811 07:42:51.849449 6566 catalog_manager.cc:4584] TS 7854366b38174865b9fa64688c58823d (127.18.72.193:44573): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08dd0096318e4215b3e4385bd2542fff: Network error: Client connection negotiation failed: client connection to 127.18.72.193:44573: connect: Connection refused (error 111)
I20260811 07:42:51.849570 6564 catalog_manager.cc:4841] TS 8135edd9ad5e4e2faaf1fdb55868daa6 (127.18.72.195:38097): tablet 08dd0096318e4215b3e4385bd2542fff (table test-table [id=76d8a8d4860d4599aedd3aead44a940f]) successfully deleted
I20260811 07:42:51.856307 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 6926
I20260811 07:42:51.864259 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 6547
2026-08-11T07:42:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1044 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:42:51 dist-test-slave-zkpd krb5kdc[7109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:42:51 dist-test-slave-zkpd krb5kdc[7109](info): set up 2 sockets
Aug 11 07:42:51 dist-test-slave-zkpd krb5kdc[7109](info): commencing operation
krb5kdc: starting...
W20260811 07:42:52.948495 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:42:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:42:53.075008 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:32797
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:32797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:53.186823 7122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:53.187112 7122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:53.187197 7122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:53.191254 7122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:53.191350 7122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:53.191395 7122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:53.191449 7122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:53.191488 7122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:53.196720 7122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:32797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:32797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:53.197746 7122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:53.198781 7122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:53.207249 7127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:53.207329 7128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:53.207237 7130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:53.207540 7122 server_base.cc:1034] running on GCE node
I20260811 07:42:53.208146 7122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:53.209201 7122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:53.210449 7122 hybrid_clock.cc:648] HybridClock initialized: now 1786434173210432 us; error 24 us; skew 500 ppm
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:53.213239 7122 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:42:53.214135 7122 webserver.cc:460] Webserver started at http://127.18.72.254:34293/ using document root <none> and password file <none>
I20260811 07:42:53.214679 7122 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:53.214740 7122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:53.214979 7122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:53.217038 7122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "8718b5c9dcb24ce2a1d581b235bb7844"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "1a61cbba0a79fd4b02ab2289d5b1ba8c"
server_key_iv: "085a7123929ca291feaef036b4bdeb10"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.217571 7122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "8718b5c9dcb24ce2a1d581b235bb7844"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "1a61cbba0a79fd4b02ab2289d5b1ba8c"
server_key_iv: "085a7123929ca291feaef036b4bdeb10"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.221239 7122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:42:53.223426 7137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.224610 7122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:53.224738 7122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "8718b5c9dcb24ce2a1d581b235bb7844"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "1a61cbba0a79fd4b02ab2289d5b1ba8c"
server_key_iv: "085a7123929ca291feaef036b4bdeb10"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.224838 7122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:53.233255 7122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:53.236263 7122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:53.236445 7122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:53.243249 7122 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:32797
I20260811 07:42:53.243271 7197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:32797 every 8 connection(s)
I20260811 07:42:53.244189 7122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:53.246901 7198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:53.251264 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 7122
I20260811 07:42:53.251434 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:42:53.251708 18723 external_mini_cluster.cc:1428] Setting key 304be1902053d761288108a3ff9b90a6
I20260811 07:42:53.252614 7198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844: Bootstrap starting.
I20260811 07:42:53.254907 7198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:53.255827 7198 log.cc:824] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:53.257917 7198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844: No bootstrap required, opened a new log
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:53.262158 7198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8718b5c9dcb24ce2a1d581b235bb7844" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 32797 } }
I20260811 07:42:53.262459 7198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:53.262558 7198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8718b5c9dcb24ce2a1d581b235bb7844, State: Initialized, Role: FOLLOWER
I20260811 07:42:53.262989 7198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8718b5c9dcb24ce2a1d581b235bb7844" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 32797 } }
I20260811 07:42:53.263108 7198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:53.263190 7198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:53.263310 7198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:53.264194 7198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8718b5c9dcb24ce2a1d581b235bb7844" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 32797 } }
I20260811 07:42:53.264531 7198 leader_election.cc:302] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8718b5c9dcb24ce2a1d581b235bb7844; no voters:
I20260811 07:42:53.264832 7198 leader_election.cc:288] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:42:53.264895 7203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:53.265090 7203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [term 1 LEADER]: Becoming Leader. State: Replica: 8718b5c9dcb24ce2a1d581b235bb7844, State: Running, Role: LEADER
I20260811 07:42:53.265421 7203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8718b5c9dcb24ce2a1d581b235bb7844" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 32797 } }
I20260811 07:42:53.266165 7198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:53.266408 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:53.253160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50300 (local address 127.18.72.254:32797)
0811 07:42:53.253542 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:53.253557 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:53.253590 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:42:53.254218 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:53.254234 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:53.254542 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:53.254805 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:53.255699 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.256927 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.258111 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.258559 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.258681 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:53.261403 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:53.261408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:53.261419 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:53.261444 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:53.263406 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:53.263824 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:53.263830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:53.263835 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:53.264010 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:53.264291 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:53.264295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:53.264297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:53.264722 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:53.264914 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:53.265215 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 07:42:53.265558 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":112,"threads_started":1}
I20260811 07:42:53.266678 7205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8718b5c9dcb24ce2a1d581b235bb7844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8718b5c9dcb24ce2a1d581b235bb7844" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 32797 } } }
I20260811 07:42:53.267452 7205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:53.267328 7204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8718b5c9dcb24ce2a1d581b235bb7844. Latest consensus state: current_term: 1 leader_uuid: "8718b5c9dcb24ce2a1d581b235bb7844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8718b5c9dcb24ce2a1d581b235bb7844" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 32797 } } }
I20260811 07:42:53.267800 7204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844 [sys.catalog]: This master's current role is: LEADER
W20260811 07:42:53.271001 7220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:42:53.271086 7220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:42:53.271155 7212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:53.271854 7212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:53.276690 7212 catalog_manager.cc:1347] Generated new cluster ID: bd48ce9654814c499fa74e026670ceea
I20260811 07:42:53.276753 7212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:53.284976 7212 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:42:53.285818 7212 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:53.292748 7212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844: Generated new TSK 0
I20260811 07:42:53.293479 7212 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.72.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
Entry for principal kudu/127.18.72.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab.
I20260811 07:42:53.330379 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:32797
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:53.444501 7226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:53.444751 7226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:53.444792 7226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:53.448844 7226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:53.448896 7226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:53.448987 7226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:42:53.454231 7226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.193.keytab
--principal=kudu/127.18.72.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:32797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:53.455224 7226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:53.456449 7226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:53.468906 7234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:53.468910 7231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:53.469090 7226 server_base.cc:1034] running on GCE node
W20260811 07:42:53.468991 7232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:53.469610 7226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:53.470189 7226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:53.471347 7226 hybrid_clock.cc:648] HybridClock initialized: now 1786434173471332 us; error 25 us; skew 500 ppm
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:53.474179 7226 init.cc:377] Logged in from keytab as kudu/127.18.72.193@KRBTEST.COM (short username kudu)
I20260811 07:42:53.475054 7226 webserver.cc:460] Webserver started at http://127.18.72.193:42531/ using document root <none> and password file <none>
I20260811 07:42:53.475610 7226 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:53.475672 7226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:53.475966 7226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:53.478075 7226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "36f277a74e75434cb8e28330be6fc018"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "7ed0083485ea4b2031f3a9c5e428aa03"
server_key_iv: "1990b53918809041528fc0aebbb641cf"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.478645 7226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "36f277a74e75434cb8e28330be6fc018"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "7ed0083485ea4b2031f3a9c5e428aa03"
server_key_iv: "1990b53918809041528fc0aebbb641cf"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.482323 7226 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 07:42:53.484649 7241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.485808 7226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:53.485944 7226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "36f277a74e75434cb8e28330be6fc018"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "7ed0083485ea4b2031f3a9c5e428aa03"
server_key_iv: "1990b53918809041528fc0aebbb641cf"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.486049 7226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:53.519513 7226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:53.522936 7226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:53.523144 7226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:53.523757 7226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:53.524633 7226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:53.524703 7226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.524773 7226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:53.524814 7226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.534060 7226 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:44049
I20260811 07:42:53.534090 7358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:44049 every 8 connection(s)
I20260811 07:42:53.534929 7226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:42:53.537858 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 7226
I20260811 07:42:53.538013 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:42:53.538371 18723 external_mini_cluster.cc:1428] Setting key 54fa221eafc0610a1bd983efce028029
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.193@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.194@KRBTEST.COM; defaulting to no policy
I20260811 07:42:53.550159 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:53.536724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:55367 (local address 127.18.72.254:32797)
0811 07:42:53.536855 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:53.536859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:53.537761 (+ 902us) server_negotiation.cc:407] Connection header received
0811 07:42:53.539530 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:53.539533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:53.539577 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:53.539639 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:53.541295 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.541805 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.543101 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.543528 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.543650 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:53.546429 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:53.546432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:53.546435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:53.546459 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:53.547878 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:53.548575 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:53.548578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:53.548579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:53.548654 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:53.549036 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:53.549039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:53.549040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:53.549206 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:53.549282 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:53.549915 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 07:42:53.550039 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:42:53.551101 7362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:53.537095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:32797 (local address 127.18.72.193:55367)
0811 07:42:53.537545 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:53.537639 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:53.539276 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:53.539847 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:53.539855 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:53.540393 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:53.541122 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:53.541146 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.541941 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.541948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:53.542965 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:53.542976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.543668 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.543670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:53.543806 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:53.544428 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:53.544444 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:53.546251 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:53.548038 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:53.548048 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:53.548064 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:53.548477 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:53.548751 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:53.548757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:53.548759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:53.548941 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:53.549319 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:53.549330 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:53.549656 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 07:42:53.549870 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 07:42:53.550241 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.194@KRBTEST.COM" created.
I20260811 07:42:53.552538 7359 heartbeater.cc:347] Connected to a master server at 127.18.72.254:32797
I20260811 07:42:53.552804 7359 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:53.553418 7359 heartbeater.cc:511] Master 127.18.72.254:32797 requested a full tablet report, sending...
I20260811 07:42:53.555008 7156 ts_manager.cc:194] Registered new tserver with Master: 36f277a74e75434cb8e28330be6fc018 (127.18.72.193:44049)
I20260811 07:42:53.556198 7156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.193@KRBTEST.COM'} at 127.18.72.193:55367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
Entry for principal kudu/127.18.72.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab.
I20260811 07:42:53.570366 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:32797
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:53.682924 7366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:53.683200 7366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:53.683272 7366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:53.687227 7366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:53.687305 7366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:53.687394 7366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:42:53.692556 7366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.194.keytab
--principal=kudu/127.18.72.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:32797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:53.693539 7366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:53.694546 7366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:53.703830 7375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:53.703773 7371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:53.703863 7366 server_base.cc:1034] running on GCE node
W20260811 07:42:53.703780 7373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:53.704432 7366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:53.704989 7366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:53.706166 7366 hybrid_clock.cc:648] HybridClock initialized: now 1786434173706138 us; error 44 us; skew 500 ppm
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:53.708952 7366 init.cc:377] Logged in from keytab as kudu/127.18.72.194@KRBTEST.COM (short username kudu)
I20260811 07:42:53.709795 7366 webserver.cc:460] Webserver started at http://127.18.72.194:38383/ using document root <none> and password file <none>
I20260811 07:42:53.710299 7366 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:53.710359 7366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:53.710516 7366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:53.712478 7366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "18cf11cee13c45fcb61fcb2c6d267daa"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "ab8abfacbc7ba9893feb0163e408413b"
server_key_iv: "dda8c7bb87d4a60ac3521037b765c6a6"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.712981 7366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "18cf11cee13c45fcb61fcb2c6d267daa"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "ab8abfacbc7ba9893feb0163e408413b"
server_key_iv: "dda8c7bb87d4a60ac3521037b765c6a6"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.716480 7366 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:42:53.718662 7382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.719790 7366 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:53.719938 7366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "18cf11cee13c45fcb61fcb2c6d267daa"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "ab8abfacbc7ba9893feb0163e408413b"
server_key_iv: "dda8c7bb87d4a60ac3521037b765c6a6"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.720034 7366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:53.738657 7366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:53.741775 7366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:53.741926 7366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:53.742420 7366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:53.743161 7366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:53.743207 7366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.743244 7366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:53.743265 7366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.752355 7366 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:45955
I20260811 07:42:53.752385 7506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:45955 every 8 connection(s)
I20260811 07:42:53.753173 7366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:42:53.756795 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 7366
I20260811 07:42:53.756922 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:42:53.757215 18723 external_mini_cluster.cc:1428] Setting key 81a09586965183a315c12b49ce226b11
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.194@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:53.767261 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:53.754917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:57887 (local address 127.18.72.254:32797)
0811 07:42:53.755046 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:53.755049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:53.755827 (+ 778us) server_negotiation.cc:407] Connection header received
0811 07:42:53.757374 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:53.757377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:53.757435 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:53.757533 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:53.758605 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.759125 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.760305 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.760695 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.760813 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:53.763293 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:53.763299 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:53.763306 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:53.763340 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:53.765085 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:53.765586 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:53.765592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:53.765597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:53.765695 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:53.766079 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:53.766084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:53.766087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:53.766319 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:53.766437 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:53.766952 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 07:42:53.767084 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:42:53.768090 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:53.755249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:32797 (local address 127.18.72.194:57887)
0811 07:42:53.755676 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:53.755718 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:53.757135 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:53.757706 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:53.757713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:53.758032 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:53.758471 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:53.758483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.759252 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.759255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:53.760186 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:53.760196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.760854 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.760858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:53.761005 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:53.761536 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:53.761550 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:53.763130 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:53.765200 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:53.765205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:53.765216 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:53.765464 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:53.765818 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:53.765823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:53.765825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:53.765981 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:53.766426 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:53.766431 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:53.766637 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 07:42:53.766867 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 07:42:53.767129 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.18.72.195@KRBTEST.COM; defaulting to no policy
I20260811 07:42:53.769170 7508 heartbeater.cc:347] Connected to a master server at 127.18.72.254:32797
I20260811 07:42:53.769397 7508 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:53.769936 7508 heartbeater.cc:511] Master 127.18.72.254:32797 requested a full tablet report, sending...
I20260811 07:42:53.770948 7156 ts_manager.cc:194] Registered new tserver with Master: 18cf11cee13c45fcb61fcb2c6d267daa (127.18.72.194:45955)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.195@KRBTEST.COM" created.
I20260811 07:42:53.771579 7156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.194@KRBTEST.COM'} at 127.18.72.194:57887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
Entry for principal kudu/127.18.72.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab.
I20260811 07:42:53.786667 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:32797
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:53.898463 7513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:53.898759 7513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:53.898869 7513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:53.902968 7513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:53.903029 7513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:53.903127 7513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:42:53.908561 7513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.195.keytab
--principal=kudu/127.18.72.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:32797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:53.909578 7513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:53.910595 7513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:53.920270 7520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:53.920287 7522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:53.920287 7519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:53.920418 7513 server_base.cc:1034] running on GCE node
I20260811 07:42:53.920982 7513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:53.921605 7513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:53.922760 7513 hybrid_clock.cc:648] HybridClock initialized: now 1786434173922730 us; error 40 us; skew 500 ppm
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:53.925603 7513 init.cc:377] Logged in from keytab as kudu/127.18.72.195@KRBTEST.COM (short username kudu)
I20260811 07:42:53.926469 7513 webserver.cc:460] Webserver started at http://127.18.72.195:45793/ using document root <none> and password file <none>
I20260811 07:42:53.927030 7513 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:53.927089 7513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:53.927332 7513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:53.929708 7513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "7fac3d0800274c1f8ec2e42c3709667a"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "f06d4973a0ca5f7e4c70ee9af30a4a90"
server_key_iv: "4314d5ad98f3aeddeef971ca27ce9acb"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.930274 7513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "7fac3d0800274c1f8ec2e42c3709667a"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "f06d4973a0ca5f7e4c70ee9af30a4a90"
server_key_iv: "4314d5ad98f3aeddeef971ca27ce9acb"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.934546 7513 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:42:53.937107 7529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.938369 7513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:42:53.938516 7513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "7fac3d0800274c1f8ec2e42c3709667a"
format_stamp: "Formatted at 2026-08-11 07:42:53 on dist-test-slave-zkpd"
server_key: "f06d4973a0ca5f7e4c70ee9af30a4a90"
server_key_iv: "4314d5ad98f3aeddeef971ca27ce9acb"
server_key_version: "encryptionkey@0"
I20260811 07:42:53.938628 7513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:53.960244 7513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:53.963122 7513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:53.963326 7513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:53.963938 7513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:53.964753 7513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:53.964830 7513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.964901 7513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:53.964952 7513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:53.974339 7513 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:46847
I20260811 07:42:53.974367 7649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:46847 every 8 connection(s)
I20260811 07:42:53.975190 7513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:42:53.983131 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 7513
I20260811 07:42:53.983299 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:42:53.983702 18723 external_mini_cluster.cc:1428] Setting key da4763598ae07554665ac4b0d92060ba
Aug 11 07:42:53 dist-test-slave-zkpd krb5kdc[7109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434173, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.195@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:42:53.989286 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:53.977039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:45957 (local address 127.18.72.254:32797)
0811 07:42:53.977180 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:53.977185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:53.978017 (+ 832us) server_negotiation.cc:407] Connection header received
0811 07:42:53.979380 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:53.979384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:53.979444 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:53.979556 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:53.980723 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.981262 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.982442 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.982663 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.982777 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:53.985670 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:53.985673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:53.985676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:53.985724 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:53.987105 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:53.987628 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:53.987631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:53.987633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:53.987720 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:53.988142 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:53.988147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:53.988149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:53.988388 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:53.988501 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:53.988998 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 07:42:53.989120 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:42:53.990092 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:53.977404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:32797 (local address 127.18.72.195:45957)
0811 07:42:53.977866 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:53.977909 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:53.979156 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:53.979737 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:53.979743 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:53.980107 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:53.980575 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:53.980589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.981399 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.981409 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:53.982285 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:53.982294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:53.982818 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:53.982822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:53.983044 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:53.983676 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:42:53.983697 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:53.985500 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:53.987238 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:53.987244 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:53.987259 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:53.987521 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:53.987823 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:42:53.987826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:42:53.987827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:42:53.988055 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:42:53.988497 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:53.988504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:53.988751 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 07:42:53.988937 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 07:42:53.989250 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":123,"threads_started":1}
I20260811 07:42:53.991252 7650 heartbeater.cc:347] Connected to a master server at 127.18.72.254:32797
I20260811 07:42:53.991466 7650 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:53.991938 7650 heartbeater.cc:511] Master 127.18.72.254:32797 requested a full tablet report, sending...
I20260811 07:42:53.992897 7156 ts_manager.cc:194] Registered new tserver with Master: 7fac3d0800274c1f8ec2e42c3709667a (127.18.72.195:46847)
I20260811 07:42:53.993445 7156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.72.195@KRBTEST.COM'} at 127.18.72.195:45957
I20260811 07:42:53.997738 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:42:54.007995 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:53.999720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50306 (local address 127.18.72.254:32797)
0811 07:42:53.999897 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:53.999921 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.000068 (+ 147us) server_negotiation.cc:407] Connection header received
0811 07:42:54.000248 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.000250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.000290 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.000376 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:54.001080 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.001585 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.002805 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.003033 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.003147 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.004712 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:54.004716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:54.004719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:54.004750 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:54.006244 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:54.006676 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:54.006680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:54.006682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:54.006767 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:54.007056 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:54.007059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:54.007061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:54.007289 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:54.007429 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.007660 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.007780 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 07:42:54.010876 7156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:42:54.014065 7156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:42:54.029009 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.022654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:44049 (local address 127.0.0.1:53794)
0811 07:42:54.023474 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:54.023520 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:54.023663 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:54.024363 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:54.024369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:54.024393 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:54.024515 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.024522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.025819 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.025825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.027166 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.027178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.028368 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.028372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.028545 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.028590 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 07:42:54.028719 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 07:42:54.028854 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"spinlock_wait_cycles":10496,"thread_start_us":57,"threads_started":1}
I20260811 07:42:54.029018 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.022830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.18.72.193:44049)
0811 07:42:54.023312 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.023329 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.023830 (+ 501us) server_negotiation.cc:407] Connection header received
0811 07:42:54.023951 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.023959 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.024173 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.024398 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:54.024670 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.025686 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.027331 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.028203 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.028369 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.028518 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.028768 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.028869 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":138,"threads_started":1}
I20260811 07:42:54.029358 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.022731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:46847 (local address 127.0.0.1:43340)
0811 07:42:54.023471 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:54.023520 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:54.023667 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:54.024749 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:54.024752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:54.024772 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:54.024866 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.024877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.026418 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.026421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.027622 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.027632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.028923 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.028927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.029034 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.029070 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 07:42:54.029161 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 07:42:54.029243 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":32,"spinlock_wait_cycles":4864,"thread_start_us":94,"threads_started":1}
I20260811 07:42:54.029825 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.023084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:45955 (local address 127.0.0.1:43864)
0811 07:42:54.023955 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:54.023971 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:54.024059 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:54.025127 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:54.025130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:54.025148 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:54.025228 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.025233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.026700 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.026707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.028002 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.028014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.029416 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.029420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.029509 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.029544 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 07:42:54.029616 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 07:42:54.029688 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":27,"thread_start_us":110,"threads_started":1}
I20260811 07:42:54.030217 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.023566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.18.72.194:45955)
0811 07:42:54.024110 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.024120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.024143 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:42:54.024767 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.024772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.024981 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.025138 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:54.025345 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.026548 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.028143 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.029015 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.029160 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.029305 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.029991 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.030086 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":152,"threads_started":1}
I20260811 07:42:54.030493 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.023514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.18.72.195:46847)
0811 07:42:54.024271 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.024284 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.024306 (+ 22us) server_negotiation.cc:407] Connection header received
0811 07:42:54.024381 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.024386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.024604 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.024762 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:54.024995 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.026182 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.027820 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.028787 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.029459 (+ 672us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.029618 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.029672 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.029783 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":168,"threads_started":1}
I20260811 07:42:54.032533 7290 tablet_service.cc:1467] Processing CreateTablet for tablet 28e07ab48ced47b1afc968f2fda6db43 (DEFAULT_TABLE table=test-table [id=dab92668e701487a80dec95e1c56a3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:54.033058 7582 tablet_service.cc:1467] Processing CreateTablet for tablet 28e07ab48ced47b1afc968f2fda6db43 (DEFAULT_TABLE table=test-table [id=dab92668e701487a80dec95e1c56a3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:54.033058 7434 tablet_service.cc:1467] Processing CreateTablet for tablet 28e07ab48ced47b1afc968f2fda6db43 (DEFAULT_TABLE table=test-table [id=dab92668e701487a80dec95e1c56a3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:54.033614 7290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e07ab48ced47b1afc968f2fda6db43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:54.034118 7582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e07ab48ced47b1afc968f2fda6db43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:54.034118 7434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e07ab48ced47b1afc968f2fda6db43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:54.040077 7668 tablet_bootstrap.cc:492] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018: Bootstrap starting.
I20260811 07:42:54.041770 7670 tablet_bootstrap.cc:492] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: Bootstrap starting.
I20260811 07:42:54.041770 7669 tablet_bootstrap.cc:492] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: Bootstrap starting.
I20260811 07:42:54.042534 7668 tablet_bootstrap.cc:654] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:54.043823 7668 log.cc:824] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:54.043988 7670 tablet_bootstrap.cc:654] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:54.043987 7669 tablet_bootstrap.cc:654] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:54.045320 7670 log.cc:824] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:54.045320 7669 log.cc:824] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:54.046316 7668 tablet_bootstrap.cc:492] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018: No bootstrap required, opened a new log
I20260811 07:42:54.046528 7668 ts_tablet_manager.cc:1397] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 07:42:54.047284 7670 tablet_bootstrap.cc:492] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: No bootstrap required, opened a new log
I20260811 07:42:54.047292 7669 tablet_bootstrap.cc:492] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: No bootstrap required, opened a new log
I20260811 07:42:54.047474 7669 ts_tablet_manager.cc:1397] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 07:42:54.047474 7670 ts_tablet_manager.cc:1397] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 07:42:54.049546 7668 raft_consensus.cc:348] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.049814 7668 raft_consensus.cc:374] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:54.049889 7668 raft_consensus.cc:729] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f277a74e75434cb8e28330be6fc018, State: Initialized, Role: FOLLOWER
I20260811 07:42:54.050359 7668 consensus_queue.cc:260] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.051414 7668 ts_tablet_manager.cc:1428] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260811 07:42:54.051473 7670 raft_consensus.cc:348] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.051473 7669 raft_consensus.cc:348] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.051746 7669 raft_consensus.cc:374] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:54.051769 7670 raft_consensus.cc:374] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:54.051823 7670 raft_consensus.cc:729] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18cf11cee13c45fcb61fcb2c6d267daa, State: Initialized, Role: FOLLOWER
I20260811 07:42:54.051843 7669 raft_consensus.cc:729] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fac3d0800274c1f8ec2e42c3709667a, State: Initialized, Role: FOLLOWER
I20260811 07:42:54.052353 7359 heartbeater.cc:503] Master 127.18.72.254:32797 was elected leader, sending a full tablet report...
I20260811 07:42:54.052296 7670 consensus_queue.cc:260] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.052296 7669 consensus_queue.cc:260] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.053253 7669 ts_tablet_manager.cc:1428] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 07:42:54.053248 7670 ts_tablet_manager.cc:1428] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 07:42:54.053593 7508 heartbeater.cc:503] Master 127.18.72.254:32797 was elected leader, sending a full tablet report...
I20260811 07:42:54.054774 7650 heartbeater.cc:503] Master 127.18.72.254:32797 was elected leader, sending a full tablet report...
I20260811 07:42:54.120261 7676 raft_consensus.cc:482] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:54.120580 7676 raft_consensus.cc:504] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.121800 7676 leader_election.cc:288] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36f277a74e75434cb8e28330be6fc018 (127.18.72.193:44049), 7fac3d0800274c1f8ec2e42c3709667a (127.18.72.195:46847)
I20260811 07:42:54.126142 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.122140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:47375 (local address 127.18.72.195:46847)
0811 07:42:54.122265 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.122270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.122291 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:42:54.122476 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.122478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.122525 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.122627 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:54.122980 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.123600 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.125015 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.125608 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.125740 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.125813 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.125990 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.126048 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:42:54.126165 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.122042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:46847 (local address 127.18.72.194:47375)
0811 07:42:54.122143 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:54.122160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:54.122278 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:54.122642 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:54.122645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:54.122668 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:54.122842 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.122851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.123739 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.123743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.124895 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.124904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.125733 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.125735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.125862 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.125889 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 07:42:54.125967 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:42:54.126034 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 07:42:54.126885 7603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e07ab48ced47b1afc968f2fda6db43" candidate_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fac3d0800274c1f8ec2e42c3709667a" is_pre_election: true
I20260811 07:42:54.126984 7677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.122042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:44049 (local address 127.18.72.194:57779)
0811 07:42:54.122797 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:54.122812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:54.123158 (+ 346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:54.123476 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:54.123479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:54.123498 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:54.123620 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.123629 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.124624 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.124626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.125734 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.125744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.126601 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.126604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.126717 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.126746 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 07:42:54.126819 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:42:54.126878 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":402,"thread_start_us":172,"threads_started":1}
I20260811 07:42:54.127050 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.122140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:57779 (local address 127.18.72.193:44049)
0811 07:42:54.122286 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.122290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.123105 (+ 815us) server_negotiation.cc:407] Connection header received
0811 07:42:54.123305 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.123308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.123354 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.123447 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:54.123740 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.124524 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.125865 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.126465 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.126598 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.126650 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.126858 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.126935 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:42:54.127233 7603 raft_consensus.cc:2454] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18cf11cee13c45fcb61fcb2c6d267daa in term 0.
I20260811 07:42:54.127719 7311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e07ab48ced47b1afc968f2fda6db43" candidate_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f277a74e75434cb8e28330be6fc018" is_pre_election: true
I20260811 07:42:54.127753 7386 leader_election.cc:302] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18cf11cee13c45fcb61fcb2c6d267daa, 7fac3d0800274c1f8ec2e42c3709667a; no voters:
I20260811 07:42:54.128057 7311 raft_consensus.cc:2454] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18cf11cee13c45fcb61fcb2c6d267daa in term 0.
I20260811 07:42:54.128059 7676 raft_consensus.cc:2781] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:42:54.128153 7676 raft_consensus.cc:482] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:42:54.128199 7676 raft_consensus.cc:3037] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:54.129310 7676 raft_consensus.cc:504] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.129637 7676 leader_election.cc:288] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [CANDIDATE]: Term 1 election: Requested vote from peers 36f277a74e75434cb8e28330be6fc018 (127.18.72.193:44049), 7fac3d0800274c1f8ec2e42c3709667a (127.18.72.195:46847)
I20260811 07:42:54.129933 7311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e07ab48ced47b1afc968f2fda6db43" candidate_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f277a74e75434cb8e28330be6fc018"
I20260811 07:42:54.130007 7603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e07ab48ced47b1afc968f2fda6db43" candidate_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fac3d0800274c1f8ec2e42c3709667a"
I20260811 07:42:54.130079 7311 raft_consensus.cc:3037] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:54.130152 7603 raft_consensus.cc:3037] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:54.131098 7311 raft_consensus.cc:2454] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18cf11cee13c45fcb61fcb2c6d267daa in term 1.
I20260811 07:42:54.131261 7603 raft_consensus.cc:2454] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18cf11cee13c45fcb61fcb2c6d267daa in term 1.
I20260811 07:42:54.131440 7385 leader_election.cc:302] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18cf11cee13c45fcb61fcb2c6d267daa, 36f277a74e75434cb8e28330be6fc018; no voters:
I20260811 07:42:54.131613 7676 raft_consensus.cc:2781] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:54.131839 7676 raft_consensus.cc:686] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 1 LEADER]: Becoming Leader. State: Replica: 18cf11cee13c45fcb61fcb2c6d267daa, State: Running, Role: LEADER
I20260811 07:42:54.132190 7676 consensus_queue.cc:237] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } }
I20260811 07:42:54.135318 7156 catalog_manager.cc:5495] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa reported cstate change: term changed from 0 to 1, leader changed from <none> to 18cf11cee13c45fcb61fcb2c6d267daa (127.18.72.194). New cstate: current_term: 1 leader_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18cf11cee13c45fcb61fcb2c6d267daa" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45955 } health_report { overall_health: HEALTHY } } }
I20260811 07:42:54.149654 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.146451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.18.72.194:45955)
0811 07:42:54.146571 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.146574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.146595 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:42:54.146685 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.146687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.146744 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.146832 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:42:54.147051 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.147533 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.148407 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.148627 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.148717 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.148956 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:42:54.149355 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:42:54.149445 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.149494 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.149540 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:42:54.155289 7603 raft_consensus.cc:1261] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 1 FOLLOWER]: Refusing update from remote peer 18cf11cee13c45fcb61fcb2c6d267daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:54.155305 7311 raft_consensus.cc:1261] T 28e07ab48ced47b1afc968f2fda6db43 P 36f277a74e75434cb8e28330be6fc018 [term 1 FOLLOWER]: Refusing update from remote peer 18cf11cee13c45fcb61fcb2c6d267daa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:54.156016 7676 consensus_queue.cc:1035] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fac3d0800274c1f8ec2e42c3709667a" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 46847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:54.156215 7678 consensus_queue.cc:1035] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [LEADER]: Connected to new peer: Peer: permanent_uuid: "36f277a74e75434cb8e28330be6fc018" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 44049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:54.163317 7682 mvcc.cc:205] Tried to move back new op lower bound from 7317234377333399552 to 7317234377248223232. Current Snapshot: MvccSnapshot[applied={T|T < 7317234377333399552}]
I20260811 07:42:54.166682 7684 mvcc.cc:205] Tried to move back new op lower bound from 7317234377333399552 to 7317234377248223232. Current Snapshot: MvccSnapshot[applied={T|T < 7317234377333399552}]
I20260811 07:42:54.171486 7155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:54.171633 7155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:54.174089 7155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8718b5c9dcb24ce2a1d581b235bb7844: Sending DeleteTablet for 3 replicas of tablet 28e07ab48ced47b1afc968f2fda6db43
I20260811 07:42:54.174865 7290 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e07ab48ced47b1afc968f2fda6db43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:54 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:53794
I20260811 07:42:54.174865 7582 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e07ab48ced47b1afc968f2fda6db43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:54 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43340
I20260811 07:42:54.175009 7434 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e07ab48ced47b1afc968f2fda6db43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:54 UTC) from {username='kudu', principal='kudu/127.18.72.254@KRBTEST.COM'} at 127.0.0.1:43864
I20260811 07:42:54.175390 7693 tablet_replica.cc:330] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: stopping tablet replica
I20260811 07:42:54.175729 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 7226
I20260811 07:42:54.175714 7693 raft_consensus.cc:2229] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:54.178068 7693 raft_consensus.cc:2258] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:54.178458 7696 tablet_replica.cc:330] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: stopping tablet replica
I20260811 07:42:54.178738 7696 raft_consensus.cc:2229] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:42:54.179068 7696 raft_consensus.cc:2258] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:54.179845 7693 ts_tablet_manager.cc:1905] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:54.180302 7696 ts_tablet_manager.cc:1905] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:54.182533 7696 ts_tablet_manager.cc:1918] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:54.182641 7696 log.cc:1196] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/28e07ab48ced47b1afc968f2fda6db43
I20260811 07:42:54.182984 7696 ts_tablet_manager.cc:1939] T 28e07ab48ced47b1afc968f2fda6db43 P 18cf11cee13c45fcb61fcb2c6d267daa: Deleting consensus metadata
I20260811 07:42:54.183463 7693 ts_tablet_manager.cc:1918] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:54.183571 7693 log.cc:1196] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/28e07ab48ced47b1afc968f2fda6db43
I20260811 07:42:54.183934 7693 ts_tablet_manager.cc:1939] T 28e07ab48ced47b1afc968f2fda6db43 P 7fac3d0800274c1f8ec2e42c3709667a: Deleting consensus metadata
I20260811 07:42:54.184095 7138 catalog_manager.cc:4841] TS 18cf11cee13c45fcb61fcb2c6d267daa (127.18.72.194:45955): tablet 28e07ab48ced47b1afc968f2fda6db43 (table test-table [id=dab92668e701487a80dec95e1c56a3b9]) successfully deleted
I20260811 07:42:54.185210 7141 catalog_manager.cc:4841] TS 7fac3d0800274c1f8ec2e42c3709667a (127.18.72.195:46847): tablet 28e07ab48ced47b1afc968f2fda6db43 (table test-table [id=dab92668e701487a80dec95e1c56a3b9]) successfully deleted
W20260811 07:42:54.187549 7140 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:44049 (error 108)
I20260811 07:42:54.187994 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 7366
I20260811 07:42:54.191164 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.190859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:44049 (local address 127.0.0.1:53804)
0811 07:42:54.191008 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:54.191085 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:44049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 07:42:54.191358 7140 catalog_manager.cc:4584] TS 36f277a74e75434cb8e28330be6fc018 (127.18.72.193:44049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28e07ab48ced47b1afc968f2fda6db43: Network error: Client connection negotiation failed: client connection to 127.18.72.193:44049: connect: Connection refused (error 111)
I20260811 07:42:54.196156 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 7513
I20260811 07:42:54.203262 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 7122
2026-08-11T07:42:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2342 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T07:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:54Z Disabled control of system clock
I20260811 07:42:54.239252 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:38489
--webserver_interface=127.18.72.254
--webserver_port=0
--builtin_ntp_servers=127.18.72.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:38489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:42:54.351671 7701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:54.351927 7701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:54.351963 7701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:54.356021 7701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:54.356070 7701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:54.356086 7701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:54.356102 7701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:54.356117 7701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:54.360667 7701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:38489
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:38489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:54.361617 7701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:54.362933 7701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:54.371363 7707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:54.371359 7706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:54.371464 7701 server_base.cc:1034] running on GCE node
W20260811 07:42:54.371359 7709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:54.372244 7701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:54.373330 7701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:54.374496 7701 hybrid_clock.cc:648] HybridClock initialized: now 1786434174374476 us; error 29 us; skew 500 ppm
I20260811 07:42:54.376274 7701 webserver.cc:460] Webserver started at http://127.18.72.254:43885/ using document root <none> and password file <none>
I20260811 07:42:54.376827 7701 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:54.376897 7701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:54.377130 7701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:54.379127 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "6441cd234717457a8fdc6cd106300ec5"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "4b626031d0cc7fd1284fd221df1da048"
server_key_iv: "d5a84d9dc91cb859126620cb96d88e6b"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.379679 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "6441cd234717457a8fdc6cd106300ec5"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "4b626031d0cc7fd1284fd221df1da048"
server_key_iv: "d5a84d9dc91cb859126620cb96d88e6b"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.383185 7701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:42:54.385615 7715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:54.386813 7701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:42:54.386955 7701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "6441cd234717457a8fdc6cd106300ec5"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "4b626031d0cc7fd1284fd221df1da048"
server_key_iv: "d5a84d9dc91cb859126620cb96d88e6b"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.387060 7701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:54.409149 7701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:54.409883 7701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:54.410084 7701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:54.417260 7701 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:38489
I20260811 07:42:54.417279 7769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:38489 every 8 connection(s)
I20260811 07:42:54.418220 7701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:54.421346 7770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:54.425146 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 7701
I20260811 07:42:54.425243 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:42:54.425527 18723 external_mini_cluster.cc:1428] Setting key 61484a1bfae655fb0265f80bf5378a62
I20260811 07:42:54.427366 7770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5: Bootstrap starting.
I20260811 07:42:54.429818 7770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:54.430619 7770 log.cc:824] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:54.432811 7770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5: No bootstrap required, opened a new log
I20260811 07:42:54.434756 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.427003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.18.72.254:38489)
0811 07:42:54.427642 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.427658 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.427693 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:42:54.428388 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.428407 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.428791 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.429062 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:54.429411 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.430452 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.431604 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.432115 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.432231 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.433075 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:54.433080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:54.433093 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:54.433470 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:54.433526 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:54.433545 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:54.433620 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.433801 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.434083 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":128,"threads_started":1}
I20260811 07:42:54.436547 7770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6441cd234717457a8fdc6cd106300ec5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38489 } }
I20260811 07:42:54.436792 7770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:54.436849 7770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6441cd234717457a8fdc6cd106300ec5, State: Initialized, Role: FOLLOWER
I20260811 07:42:54.437258 7770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6441cd234717457a8fdc6cd106300ec5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38489 } }
I20260811 07:42:54.437424 7770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:54.437502 7770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:54.437628 7770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:54.438488 7770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6441cd234717457a8fdc6cd106300ec5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38489 } }
I20260811 07:42:54.438831 7770 leader_election.cc:302] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6441cd234717457a8fdc6cd106300ec5; no voters:
I20260811 07:42:54.439127 7770 leader_election.cc:288] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:42:54.439227 7775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:54.439476 7775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [term 1 LEADER]: Becoming Leader. State: Replica: 6441cd234717457a8fdc6cd106300ec5, State: Running, Role: LEADER
I20260811 07:42:54.439779 7775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6441cd234717457a8fdc6cd106300ec5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38489 } }
I20260811 07:42:54.440538 7770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:54.441601 7777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6441cd234717457a8fdc6cd106300ec5. Latest consensus state: current_term: 1 leader_uuid: "6441cd234717457a8fdc6cd106300ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6441cd234717457a8fdc6cd106300ec5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38489 } } }
I20260811 07:42:54.441574 7776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6441cd234717457a8fdc6cd106300ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6441cd234717457a8fdc6cd106300ec5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38489 } } }
I20260811 07:42:54.441735 7777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:54.441740 7776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:54.442128 7785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:54.444885 7785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:54.450321 7785 catalog_manager.cc:1347] Generated new cluster ID: ad16478402e3463587135b4f1d952feb
I20260811 07:42:54.450398 7785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:54.494311 7785 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:42:54.495316 7785 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:54.501293 7785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5: Generated new TSK 0
I20260811 07:42:54.501992 7785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:42:54.508121 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.193:0
--local_ip_for_outbound_sockets=127.18.72.193
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:38489
--builtin_ntp_servers=127.18.72.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:42:54.623199 7796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:54.623467 7796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:54.623522 7796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:54.627496 7796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:54.627547 7796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:54.627595 7796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.193
I20260811 07:42:54.632519 7796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.72.193
--webserver_port=0
--tserver_master_addrs=127.18.72.254:38489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.193
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:54.633518 7796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:54.634850 7796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:54.644398 7802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:54.644415 7801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:54.644537 7796 server_base.cc:1034] running on GCE node
W20260811 07:42:54.644410 7804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:54.645045 7796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:54.645654 7796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:54.646862 7796 hybrid_clock.cc:648] HybridClock initialized: now 1786434174646829 us; error 45 us; skew 500 ppm
I20260811 07:42:54.648669 7796 webserver.cc:460] Webserver started at http://127.18.72.193:32789/ using document root <none> and password file <none>
I20260811 07:42:54.649236 7796 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:54.649336 7796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:54.649571 7796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:54.651572 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/data/instance:
uuid: "1bd517b2d58042ef80c3bf433d2eb655"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "130c58c7fa0bbcb3f8fa652abfe62d5f"
server_key_iv: "e70938d9007e998ce3f3bfc01a373d33"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.652149 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance:
uuid: "1bd517b2d58042ef80c3bf433d2eb655"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "130c58c7fa0bbcb3f8fa652abfe62d5f"
server_key_iv: "e70938d9007e998ce3f3bfc01a373d33"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.655948 7796 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 07:42:54.658236 7810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:54.659466 7796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:54.659593 7796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/wal
uuid: "1bd517b2d58042ef80c3bf433d2eb655"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "130c58c7fa0bbcb3f8fa652abfe62d5f"
server_key_iv: "e70938d9007e998ce3f3bfc01a373d33"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.659691 7796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:54.673624 7796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:54.674259 7796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:54.674451 7796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:54.675052 7796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:54.676030 7796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:54.676079 7796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:54.676141 7796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:54.676172 7796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:54.684832 7796 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.193:46875
I20260811 07:42:54.684950 7930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.193:46875 every 8 connection(s)
I20260811 07:42:54.685755 7796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/data/info.pb
I20260811 07:42:54.695111 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.687652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.193:49421 (local address 127.18.72.254:38489)
0811 07:42:54.687802 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.687806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.688856 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 07:42:54.690320 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.690323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.690383 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.690488 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:54.691369 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.691987 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.693128 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.693368 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.693481 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.694602 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:54.694606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:54.694608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:54.694622 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:54.694649 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:54.694657 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:54.694736 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.694907 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.694969 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:42:54.695485 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 7796
I20260811 07:42:54.695616 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-0/wal/instance
I20260811 07:42:54.695938 18723 external_mini_cluster.cc:1428] Setting key 392672edd0219699d2d04f0095cc0775
I20260811 07:42:54.696777 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.688058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:38489 (local address 127.18.72.193:49421)
0811 07:42:54.688700 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:54.688738 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:54.690086 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:54.690662 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:54.690669 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:54.690754 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:54.691219 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.691231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.692114 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.692122 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.692994 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.693005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.693534 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.693538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.693719 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.694342 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:54.694344 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:54.694383 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:54.694445 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:54.694750 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:54.694762 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:54.694769 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 07:42:54.694900 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 07:42:54.695171 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260811 07:42:54.697927 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.194:0
--local_ip_for_outbound_sockets=127.18.72.194
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:38489
--builtin_ntp_servers=127.18.72.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:42:54.698019 7931 heartbeater.cc:347] Connected to a master server at 127.18.72.254:38489
I20260811 07:42:54.698252 7931 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:54.698753 7931 heartbeater.cc:511] Master 127.18.72.254:38489 requested a full tablet report, sending...
I20260811 07:42:54.700274 7732 ts_manager.cc:194] Registered new tserver with Master: 1bd517b2d58042ef80c3bf433d2eb655 (127.18.72.193:46875)
I20260811 07:42:54.701402 7732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.193:49421
W20260811 07:42:54.809298 7934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:54.809516 7934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:54.809551 7934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:54.813359 7934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:54.813396 7934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:54.813445 7934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.194
I20260811 07:42:54.817919 7934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.72.194
--webserver_port=0
--tserver_master_addrs=127.18.72.254:38489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.194
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:54.818775 7934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:54.820013 7934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:54.828874 7942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:54.829057 7934 server_base.cc:1034] running on GCE node
W20260811 07:42:54.828926 7939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:54.828871 7940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:54.829746 7934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:54.830403 7934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:54.831585 7934 hybrid_clock.cc:648] HybridClock initialized: now 1786434174831567 us; error 30 us; skew 500 ppm
I20260811 07:42:54.837042 7934 webserver.cc:460] Webserver started at http://127.18.72.194:44333/ using document root <none> and password file <none>
I20260811 07:42:54.837622 7934 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:54.837719 7934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:54.837952 7934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:54.840036 7934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/data/instance:
uuid: "50aa42393adb4c19a62c8cf2f1510a77"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "6e066b2f92efbf0f0c1863a4c43c1f31"
server_key_iv: "1fd3ab5490f7a705c84904e7b633b15a"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.840647 7934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance:
uuid: "50aa42393adb4c19a62c8cf2f1510a77"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "6e066b2f92efbf0f0c1863a4c43c1f31"
server_key_iv: "1fd3ab5490f7a705c84904e7b633b15a"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.844451 7934 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 07:42:54.846741 7948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:54.847983 7934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:42:54.848109 7934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/wal
uuid: "50aa42393adb4c19a62c8cf2f1510a77"
format_stamp: "Formatted at 2026-08-11 07:42:54 on dist-test-slave-zkpd"
server_key: "6e066b2f92efbf0f0c1863a4c43c1f31"
server_key_iv: "1fd3ab5490f7a705c84904e7b633b15a"
server_key_version: "encryptionkey@0"
I20260811 07:42:54.848213 7934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:54.861959 7934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:54.862553 7934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:54.862739 7934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:54.863327 7934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:54.864300 7934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:54.864377 7934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:54.864445 7934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:54.864496 7934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:54.873042 7934 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.194:45991
I20260811 07:42:54.873076 8066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.194:45991 every 8 connection(s)
I20260811 07:42:54.873914 7934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/data/info.pb
I20260811 07:42:54.874514 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 7934
I20260811 07:42:54.874647 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/wal/instance
I20260811 07:42:54.874984 18723 external_mini_cluster.cc:1428] Setting key 442c4105b8c595252632498eee16351b
I20260811 07:42:54.877239 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.72.195:0
--local_ip_for_outbound_sockets=127.18.72.195
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:38489
--builtin_ntp_servers=127.18.72.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:42:54.884102 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.875815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.194:52457 (local address 127.18.72.254:38489)
0811 07:42:54.875975 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:54.875979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:54.877079 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 07:42:54.878556 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:54.878560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:54.878616 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:54.878694 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:54.879554 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.880228 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.881961 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.882179 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.882267 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.883572 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:54.883576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:54.883578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:54.883591 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:54.883610 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:54.883617 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:54.883672 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:54.883885 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 07:42:54.883990 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:42:54.885372 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:54.876263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:38489 (local address 127.18.72.194:52457)
0811 07:42:54.876941 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:54.876981 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:54.878351 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:54.878866 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:54.878873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:54.878953 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:54.879413 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.879426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.880357 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.880364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.881838 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:54.881853 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:54.882349 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:54.882356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:54.882577 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:54.883235 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:54.883240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:54.883300 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:54.883396 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:54.883706 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:54.883719 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:54.883728 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 07:42:54.883883 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 07:42:54.884232 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
I20260811 07:42:54.886587 8067 heartbeater.cc:347] Connected to a master server at 127.18.72.254:38489
I20260811 07:42:54.886822 8067 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:54.887360 8067 heartbeater.cc:511] Master 127.18.72.254:38489 requested a full tablet report, sending...
I20260811 07:42:54.888447 7732 ts_manager.cc:194] Registered new tserver with Master: 50aa42393adb4c19a62c8cf2f1510a77 (127.18.72.194:45991)
I20260811 07:42:54.889014 7732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.194:52457
W20260811 07:42:54.991923 8070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:54.992200 8070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:54.992275 8070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:54.996204 8070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:54.996286 8070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:54.996373 8070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.72.195
I20260811 07:42:55.001161 8070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.72.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.72.195
--webserver_port=0
--tserver_master_addrs=127.18.72.254:38489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.72.195
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:55.002228 8070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:55.003628 8070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:55.013093 8079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:55.013165 8075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:55.013167 8076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:55.013312 8070 server_base.cc:1034] running on GCE node
I20260811 07:42:55.013839 8070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:55.014396 8070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:55.015609 8070 hybrid_clock.cc:648] HybridClock initialized: now 1786434175015568 us; error 49 us; skew 500 ppm
I20260811 07:42:55.017414 8070 webserver.cc:460] Webserver started at http://127.18.72.195:39739/ using document root <none> and password file <none>
I20260811 07:42:55.017983 8070 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:55.018072 8070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:55.018297 8070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:55.020354 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/data/instance:
uuid: "03425514eef34531acfdc7070d8e28dd"
format_stamp: "Formatted at 2026-08-11 07:42:55 on dist-test-slave-zkpd"
server_key: "4f80de4dc04b5d578c3eb9436de7c8ce"
server_key_iv: "e60122524cb9bb41a0434a5e32894d8f"
server_key_version: "encryptionkey@0"
I20260811 07:42:55.020923 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance:
uuid: "03425514eef34531acfdc7070d8e28dd"
format_stamp: "Formatted at 2026-08-11 07:42:55 on dist-test-slave-zkpd"
server_key: "4f80de4dc04b5d578c3eb9436de7c8ce"
server_key_iv: "e60122524cb9bb41a0434a5e32894d8f"
server_key_version: "encryptionkey@0"
I20260811 07:42:55.024758 8070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 07:42:55.027246 8085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:55.028478 8070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:42:55.028607 8070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/wal
uuid: "03425514eef34531acfdc7070d8e28dd"
format_stamp: "Formatted at 2026-08-11 07:42:55 on dist-test-slave-zkpd"
server_key: "4f80de4dc04b5d578c3eb9436de7c8ce"
server_key_iv: "e60122524cb9bb41a0434a5e32894d8f"
server_key_version: "encryptionkey@0"
I20260811 07:42:55.028712 8070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:55.039388 8070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:55.040030 8070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:55.040197 8070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:55.040723 8070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:42:55.041721 8070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:42:55.041790 8070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:55.041883 8070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:42:55.041918 8070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:55.050691 8070 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.195:45553
I20260811 07:42:55.050731 8203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.195:45553 every 8 connection(s)
I20260811 07:42:55.051564 8070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/data/info.pb
I20260811 07:42:55.055186 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 8070
I20260811 07:42:55.055321 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/wal/instance
I20260811 07:42:55.055644 18723 external_mini_cluster.cc:1428] Setting key 65aaf467ea61777da614936947cde2e4
I20260811 07:42:55.060950 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.053258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:33691 (local address 127.18.72.254:38489)
0811 07:42:55.053394 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:55.053398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:55.054488 (+ 1090us) server_negotiation.cc:407] Connection header received
0811 07:42:55.056308 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:55.056311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:55.056386 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:55.056467 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:55.057533 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.058184 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.059283 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.059495 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.059568 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.060572 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:55.060574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:55.060576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:55.060588 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:55.060605 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:55.060612 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:55.060669 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:55.060801 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 07:42:55.060847 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:42:55.061739 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.053736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:38489 (local address 127.18.72.195:33691)
0811 07:42:55.054373 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:55.054410 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:55.056072 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:55.056661 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:55.056671 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:55.056773 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:42:55.057395 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.057408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.058301 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.058304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.059181 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.059191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.059618 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.059620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.059758 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.060346 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:42:55.060347 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:42:55.060383 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:42:55.060443 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:42:55.060683 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:42:55.060690 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:42:55.060695 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:42:55.060799 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 07:42:55.061003 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":101,"threads_started":1}
I20260811 07:42:55.062865 8204 heartbeater.cc:347] Connected to a master server at 127.18.72.254:38489
I20260811 07:42:55.063056 8204 heartbeater.cc:464] Registering TS with master...
I20260811 07:42:55.063520 8204 heartbeater.cc:511] Master 127.18.72.254:38489 requested a full tablet report, sending...
I20260811 07:42:55.064509 7732 ts_manager.cc:194] Registered new tserver with Master: 03425514eef34531acfdc7070d8e28dd (127.18.72.195:45553)
I20260811 07:42:55.065055 7732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.72.195:33691
I20260811 07:42:55.069396 18723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:42:55.074916 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.071202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.18.72.254:38489)
0811 07:42:55.071332 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:55.071335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:55.071534 (+ 199us) server_negotiation.cc:407] Connection header received
0811 07:42:55.071678 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:55.071681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:55.071720 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:55.071806 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:55.072100 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.072598 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.073424 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.073644 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.073788 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.074492 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:55.074496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:55.074498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:55.074510 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:55.074534 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:55.074541 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:55.074616 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:55.074731 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 07:42:55.074789 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:42:55.077425 7732 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:42:55.080413 7732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:42:55.094522 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.088546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.18.72.193:46875)
0811 07:42:55.089328 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:55.089343 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:55.089371 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:42:55.089454 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:55.089459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:55.089670 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:55.089847 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:55.090203 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.091377 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.092865 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.093654 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.093790 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.094008 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:55.094233 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 07:42:55.094342 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":127,"threads_started":1}
I20260811 07:42:55.094537 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.088684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.195:45553 (local address 127.0.0.1:45082)
0811 07:42:55.088951 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:55.088966 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:55.089050 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:55.090385 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:55.090387 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:55.090405 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:55.090488 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.090496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.091580 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.091583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.092888 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.092899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.093934 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.093938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.094112 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.094167 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 07:42:55.094224 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 07:42:55.094378 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":82,"thread_start_us":98,"threads_started":1}
I20260811 07:42:55.094682 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.089524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.18.72.194:45991)
0811 07:42:55.089942 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:55.089953 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:55.089978 (+ 25us) server_negotiation.cc:407] Connection header received
0811 07:42:55.090060 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:55.090064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:55.090317 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:55.090513 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:55.090758 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.091861 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.093191 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.093712 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.093846 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.094027 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:55.094410 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 07:42:55.094511 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":134,"threads_started":1}
I20260811 07:42:55.094519 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.089065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:45991 (local address 127.0.0.1:37644)
0811 07:42:55.089624 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:55.089637 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:55.089719 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:55.090547 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:55.090552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:55.090572 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:55.090667 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.090673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.092004 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.092007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.093057 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.093091 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.094004 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.094007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.094179 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.094206 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 07:42:55.094279 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:42:55.094355 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":53,"thread_start_us":121,"threads_started":1}
I20260811 07:42:55.095026 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.088359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:46875 (local address 127.0.0.1:42508)
0811 07:42:55.088697 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:55.088749 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:55.088871 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:55.089865 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:55.089871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:55.089893 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:55.090072 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.090078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.091493 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.091496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.092736 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.092746 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.093803 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.093810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.093915 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.094068 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 07:42:55.094181 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 07:42:55.094857 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":689,"thread_start_us":89,"threads_started":1}
I20260811 07:42:55.095497 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.089147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.18.72.195:45553)
0811 07:42:55.089907 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:55.089919 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:55.089941 (+ 22us) server_negotiation.cc:407] Connection header received
0811 07:42:55.090013 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:55.090018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:55.090240 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:55.090407 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:55.090615 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.091484 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.093006 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.093798 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.094567 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.095196 (+ 629us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:55.095247 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 07:42:55.095338 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":151,"threads_started":1}
I20260811 07:42:55.097733 7863 tablet_service.cc:1467] Processing CreateTablet for tablet 1e87f13838c547cf80f058d207f9bc54 (DEFAULT_TABLE table=test-table [id=685b3a23aa4045d7a8e0a32fe6c9a408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:55.097880 8133 tablet_service.cc:1467] Processing CreateTablet for tablet 1e87f13838c547cf80f058d207f9bc54 (DEFAULT_TABLE table=test-table [id=685b3a23aa4045d7a8e0a32fe6c9a408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:55.097880 7999 tablet_service.cc:1467] Processing CreateTablet for tablet 1e87f13838c547cf80f058d207f9bc54 (DEFAULT_TABLE table=test-table [id=685b3a23aa4045d7a8e0a32fe6c9a408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:42:55.098749 7863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e87f13838c547cf80f058d207f9bc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:55.098874 8133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e87f13838c547cf80f058d207f9bc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:55.098874 7999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e87f13838c547cf80f058d207f9bc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:55.104854 8222 tablet_bootstrap.cc:492] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655: Bootstrap starting.
I20260811 07:42:55.106199 8223 tablet_bootstrap.cc:492] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Bootstrap starting.
I20260811 07:42:55.107290 8222 tablet_bootstrap.cc:654] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:55.108224 8224 tablet_bootstrap.cc:492] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Bootstrap starting.
I20260811 07:42:55.108229 8222 log.cc:824] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:55.108297 8223 tablet_bootstrap.cc:654] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:55.109076 8223 log.cc:824] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:55.110291 8224 tablet_bootstrap.cc:654] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:55.110467 8222 tablet_bootstrap.cc:492] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655: No bootstrap required, opened a new log
I20260811 07:42:55.110667 8222 ts_tablet_manager.cc:1397] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 07:42:55.110973 8223 tablet_bootstrap.cc:492] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: No bootstrap required, opened a new log
I20260811 07:42:55.111104 8224 log.cc:824] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:55.111168 8223 ts_tablet_manager.cc:1397] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:42:55.113078 8224 tablet_bootstrap.cc:492] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: No bootstrap required, opened a new log
I20260811 07:42:55.113255 8224 ts_tablet_manager.cc:1397] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 07:42:55.114470 8223 raft_consensus.cc:348] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.114698 8223 raft_consensus.cc:374] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:55.114751 8223 raft_consensus.cc:729] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50aa42393adb4c19a62c8cf2f1510a77, State: Initialized, Role: FOLLOWER
I20260811 07:42:55.114975 8222 raft_consensus.cc:348] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.115228 8222 raft_consensus.cc:374] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:55.115294 8222 raft_consensus.cc:729] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd517b2d58042ef80c3bf433d2eb655, State: Initialized, Role: FOLLOWER
I20260811 07:42:55.115221 8223 consensus_queue.cc:260] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.115727 8222 consensus_queue.cc:260] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.116276 8223 ts_tablet_manager.cc:1428] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 07:42:55.116626 8067 heartbeater.cc:503] Master 127.18.72.254:38489 was elected leader, sending a full tablet report...
I20260811 07:42:55.116667 8222 ts_tablet_manager.cc:1428] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 07:42:55.117025 7931 heartbeater.cc:503] Master 127.18.72.254:38489 was elected leader, sending a full tablet report...
I20260811 07:42:55.117308 8224 raft_consensus.cc:348] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.117475 8224 raft_consensus.cc:374] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:55.117503 8224 raft_consensus.cc:729] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03425514eef34531acfdc7070d8e28dd, State: Initialized, Role: FOLLOWER
I20260811 07:42:55.117895 8224 consensus_queue.cc:260] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.119136 8204 heartbeater.cc:503] Master 127.18.72.254:38489 was elected leader, sending a full tablet report...
I20260811 07:42:55.119637 8224 ts_tablet_manager.cc:1428] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
W20260811 07:42:55.124888 8068 tablet.cc:2369] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:42:55.186929 7932 tablet.cc:2369] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:42:55.302543 8205 tablet.cc:2369] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:42:55.523279 8230 raft_consensus.cc:482] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:42:55.523486 8230 raft_consensus.cc:504] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.525014 8230 leader_election.cc:288] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bd517b2d58042ef80c3bf433d2eb655 (127.18.72.193:46875), 50aa42393adb4c19a62c8cf2f1510a77 (127.18.72.194:45991)
I20260811 07:42:55.529033 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.525226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:46875 (local address 127.18.72.195:40539)
0811 07:42:55.525364 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:55.525380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:55.525488 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:55.525863 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:55.525866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:55.525891 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:55.526045 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.526054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.527015 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.527019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.527880 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.527914 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.528566 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.528571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.528681 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.528752 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 07:42:55.528821 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 07:42:55.528877 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":24}
I20260811 07:42:55.529070 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.525450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:40539 (local address 127.18.72.193:46875)
0811 07:42:55.525604 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:55.525609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:55.525625 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:42:55.525692 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:55.525695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:55.525756 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:55.525860 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:55.526171 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.526889 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.528007 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.528450 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.528542 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.528579 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:55.528869 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 07:42:55.528950 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 07:42:55.529161 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.525660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.72.195:58287 (local address 127.18.72.194:45991)
0811 07:42:55.525819 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:55.525826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:55.525978 (+ 152us) server_negotiation.cc:407] Connection header received
0811 07:42:55.526093 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:55.526098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:55.526159 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:55.526227 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:42:55.526534 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.527070 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.528137 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.528549 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.528664 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.528711 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:55.528967 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 07:42:55.529033 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 07:42:55.529182 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.525549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.194:45991 (local address 127.18.72.195:58287)
0811 07:42:55.525880 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:55.525895 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:42:55.525994 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:42:55.526264 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:42:55.526267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:42:55.526285 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:42:55.526423 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.526430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.527178 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.527181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.528013 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:42:55.528023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.528689 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.528693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:42:55.528832 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.528859 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 07:42:55.528934 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 07:42:55.528998 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":18,"thread_start_us":136,"threads_started":1}
I20260811 07:42:55.529758 7883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e87f13838c547cf80f058d207f9bc54" candidate_uuid: "03425514eef34531acfdc7070d8e28dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bd517b2d58042ef80c3bf433d2eb655" is_pre_election: true
I20260811 07:42:55.529769 8019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e87f13838c547cf80f058d207f9bc54" candidate_uuid: "03425514eef34531acfdc7070d8e28dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50aa42393adb4c19a62c8cf2f1510a77" is_pre_election: true
I20260811 07:42:55.530055 8019 raft_consensus.cc:2454] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03425514eef34531acfdc7070d8e28dd in term 0.
I20260811 07:42:55.530071 7883 raft_consensus.cc:2454] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03425514eef34531acfdc7070d8e28dd in term 0.
I20260811 07:42:55.530546 8089 leader_election.cc:302] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03425514eef34531acfdc7070d8e28dd, 1bd517b2d58042ef80c3bf433d2eb655; no voters:
I20260811 07:42:55.530774 8230 raft_consensus.cc:2781] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:42:55.530853 8230 raft_consensus.cc:482] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:42:55.530884 8230 raft_consensus.cc:3037] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:55.532040 8230 raft_consensus.cc:504] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.532369 8230 leader_election.cc:288] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [CANDIDATE]: Term 1 election: Requested vote from peers 1bd517b2d58042ef80c3bf433d2eb655 (127.18.72.193:46875), 50aa42393adb4c19a62c8cf2f1510a77 (127.18.72.194:45991)
I20260811 07:42:55.532747 7883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e87f13838c547cf80f058d207f9bc54" candidate_uuid: "03425514eef34531acfdc7070d8e28dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bd517b2d58042ef80c3bf433d2eb655"
I20260811 07:42:55.532799 8019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e87f13838c547cf80f058d207f9bc54" candidate_uuid: "03425514eef34531acfdc7070d8e28dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50aa42393adb4c19a62c8cf2f1510a77"
I20260811 07:42:55.532893 7883 raft_consensus.cc:3037] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:55.532922 8019 raft_consensus.cc:3037] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:55.534201 7883 raft_consensus.cc:2454] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03425514eef34531acfdc7070d8e28dd in term 1.
I20260811 07:42:55.534272 8019 raft_consensus.cc:2454] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03425514eef34531acfdc7070d8e28dd in term 1.
I20260811 07:42:55.534662 8089 leader_election.cc:302] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03425514eef34531acfdc7070d8e28dd, 50aa42393adb4c19a62c8cf2f1510a77; no voters:
I20260811 07:42:55.534984 8230 raft_consensus.cc:2781] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:55.535175 8230 raft_consensus.cc:686] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 1 LEADER]: Becoming Leader. State: Replica: 03425514eef34531acfdc7070d8e28dd, State: Running, Role: LEADER
I20260811 07:42:55.535558 8230 consensus_queue.cc:237] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } }
I20260811 07:42:55.538689 7730 catalog_manager.cc:5495] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 03425514eef34531acfdc7070d8e28dd (127.18.72.195). New cstate: current_term: 1 leader_uuid: "03425514eef34531acfdc7070d8e28dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03425514eef34531acfdc7070d8e28dd" member_type: VOTER last_known_addr { host: "127.18.72.195" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260811 07:42:55.652428 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.648475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45084 (local address 127.18.72.195:45553)
0811 07:42:55.648717 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:55.648720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:55.648802 (+ 82us) server_negotiation.cc:407] Connection header received
0811 07:42:55.648962 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:55.648964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:55.649024 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:55.649139 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:42:55.649428 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.649963 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.650823 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:55.651036 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:55.651108 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:55.651385 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:42:55.651989 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:42:55.652094 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:55.652165 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 07:42:55.652210 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":66,"threads_started":1}
I20260811 07:42:55.658459 8019 raft_consensus.cc:1261] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 1 FOLLOWER]: Refusing update from remote peer 03425514eef34531acfdc7070d8e28dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:55.658474 7883 raft_consensus.cc:1261] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 1 FOLLOWER]: Refusing update from remote peer 03425514eef34531acfdc7070d8e28dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:42:55.659193 8230 consensus_queue.cc:1035] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bd517b2d58042ef80c3bf433d2eb655" member_type: VOTER last_known_addr { host: "127.18.72.193" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:55.659436 8232 consensus_queue.cc:1035] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "50aa42393adb4c19a62c8cf2f1510a77" member_type: VOTER last_known_addr { host: "127.18.72.194" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:42:55.666445 8237 mvcc.cc:205] Tried to move back new op lower bound from 7317234383489110016 to 7317234382996549632. Current Snapshot: MvccSnapshot[applied={T|T < 7317234383489110016}]
I20260811 07:42:55.666522 8241 mvcc.cc:205] Tried to move back new op lower bound from 7317234383489110016 to 7317234382996549632. Current Snapshot: MvccSnapshot[applied={T|T < 7317234383489110016}]
I20260811 07:42:55.674933 7730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:55.675076 7730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:42:55.677452 7730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6441cd234717457a8fdc6cd106300ec5: Sending DeleteTablet for 3 replicas of tablet 1e87f13838c547cf80f058d207f9bc54
I20260811 07:42:55.678308 8133 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e87f13838c547cf80f058d207f9bc54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:55 UTC) from {username='slave'} at 127.0.0.1:45082
I20260811 07:42:55.678485 7863 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e87f13838c547cf80f058d207f9bc54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:55 UTC) from {username='slave'} at 127.0.0.1:42508
I20260811 07:42:55.678977 8251 tablet_replica.cc:330] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655: stopping tablet replica
I20260811 07:42:55.679023 7999 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e87f13838c547cf80f058d207f9bc54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:42:55 UTC) from {username='slave'} at 127.0.0.1:37644
I20260811 07:42:55.679387 8251 raft_consensus.cc:2229] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:55.679533 8252 tablet_replica.cc:330] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: stopping tablet replica
I20260811 07:42:55.679589 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 7796
I20260811 07:42:55.679730 8251 raft_consensus.cc:2258] T 1e87f13838c547cf80f058d207f9bc54 P 1bd517b2d58042ef80c3bf433d2eb655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:55.680698 8252 raft_consensus.cc:2229] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:42:55.681032 8252 raft_consensus.cc:2258] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:55.683501 8252 ts_tablet_manager.cc:1905] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:42:55.686407 8252 ts_tablet_manager.cc:1918] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:55.686517 8252 log.cc:1196] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-1/wal/wals/1e87f13838c547cf80f058d207f9bc54
I20260811 07:42:55.686832 8252 ts_tablet_manager.cc:1939] T 1e87f13838c547cf80f058d207f9bc54 P 50aa42393adb4c19a62c8cf2f1510a77: Deleting consensus metadata
I20260811 07:42:55.688115 7719 catalog_manager.cc:4841] TS 50aa42393adb4c19a62c8cf2f1510a77 (127.18.72.194:45991): tablet 1e87f13838c547cf80f058d207f9bc54 (table test-table [id=685b3a23aa4045d7a8e0a32fe6c9a408]) successfully deleted
I20260811 07:42:55.690119 8249 tablet_replica.cc:330] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: stopping tablet replica
I20260811 07:42:55.690462 8249 raft_consensus.cc:2229] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:42:55.690867 8249 raft_consensus.cc:2258] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:42:55.692363 8249 ts_tablet_manager.cc:1905] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 07:42:55.694116 7719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.72.193:46875 (error 108)
I20260811 07:42:55.694175 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 7934
I20260811 07:42:55.695885 8254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:55.695466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.193:46875 (local address 127.0.0.1:42510)
0811 07:42:55.695699 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 07:42:55.695783 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.193:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
W20260811 07:42:55.696952 7719 catalog_manager.cc:4584] TS 1bd517b2d58042ef80c3bf433d2eb655 (127.18.72.193:46875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e87f13838c547cf80f058d207f9bc54: Network error: Client connection negotiation failed: client connection to 127.18.72.193:46875: connect: Connection refused (error 111)
I20260811 07:42:55.702265 8249 ts_tablet_manager.cc:1918] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:42:55.702380 8249 log.cc:1196] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Deleting WAL directory at /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786433920963264-18723-0/minicluster-data/ts-2/wal/wals/1e87f13838c547cf80f058d207f9bc54
I20260811 07:42:55.702867 8249 ts_tablet_manager.cc:1939] T 1e87f13838c547cf80f058d207f9bc54 P 03425514eef34531acfdc7070d8e28dd: Deleting consensus metadata
I20260811 07:42:55.703014 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 8070
I20260811 07:42:55.712038 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 7701
2026-08-11T07:42:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1504 ms)
[----------] 6 tests from EncryptionPolicyTest (11018 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:42:55 dist-test-slave-zkpd krb5kdc[8257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:42:55 dist-test-slave-zkpd krb5kdc[8257](info): set up 2 sockets
Aug 11 07:42:55 dist-test-slave-zkpd krb5kdc[8257](info): commencing operation
krb5kdc: starting...
W20260811 07:42:56.783507 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:42:56 dist-test-slave-zkpd krb5kdc[8257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:42:56.908746 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:57.019480 8271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:57.019729 8271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:57.019769 8271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:57.023592 8271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:57.023648 8271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:57.023667 8271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:57.023684 8271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:57.023698 8271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:57.028815 8271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:57.029832 8271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:57.030970 8271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:57.039386 8276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:57.039358 8278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:57.039414 8271 server_base.cc:1034] running on GCE node
W20260811 07:42:57.039376 8280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:57.040292 8271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:57.041337 8271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:57.042495 8271 hybrid_clock.cc:648] HybridClock initialized: now 1786434177042467 us; error 42 us; skew 500 ppm
Aug 11 07:42:57 dist-test-slave-zkpd krb5kdc[8257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434177, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:57.045276 8271 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:42:57.046103 8271 webserver.cc:460] Webserver started at http://127.0.0.1:36113/ using document root <none> and password file <none>
I20260811 07:42:57.046657 8271 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:57.046717 8271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:57.046958 8271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:57.049060 8271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "788571083ebf4727b4d5c1d3e94ee05a"
format_stamp: "Formatted at 2026-08-11 07:42:57 on dist-test-slave-zkpd"
server_key: "623f1340f3fe8bff0ee05e8bec98f661"
server_key_iv: "99734aa92c405fc93a496ecf85e20c58"
server_key_version: "encryptionkey@0"
I20260811 07:42:57.049612 8271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "788571083ebf4727b4d5c1d3e94ee05a"
format_stamp: "Formatted at 2026-08-11 07:42:57 on dist-test-slave-zkpd"
server_key: "623f1340f3fe8bff0ee05e8bec98f661"
server_key_iv: "99734aa92c405fc93a496ecf85e20c58"
server_key_version: "encryptionkey@0"
I20260811 07:42:57.053205 8271 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:42:57.055616 8287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:57.056804 8271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:57.056948 8271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "788571083ebf4727b4d5c1d3e94ee05a"
format_stamp: "Formatted at 2026-08-11 07:42:57 on dist-test-slave-zkpd"
server_key: "623f1340f3fe8bff0ee05e8bec98f661"
server_key_iv: "99734aa92c405fc93a496ecf85e20c58"
server_key_version: "encryptionkey@0"
I20260811 07:42:57.057053 8271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:57.096925 8271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:57.099997 8271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:57.100196 8271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:57.107251 8271 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34427
I20260811 07:42:57.107316 8341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34427 every 8 connection(s)
I20260811 07:42:57.108261 8271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:57.111078 8343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:57.115157 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 8271
I20260811 07:42:57.115334 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:42:57.115623 18723 external_mini_cluster.cc:1428] Setting key 4815396ad9d4a1d524ca74a1c6b2dc4b
I20260811 07:42:57.117013 8343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a: Bootstrap starting.
I20260811 07:42:57.119372 8343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:57.120223 8343 log.cc:824] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:57.122000 8343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a: No bootstrap required, opened a new log
Aug 11 07:42:57 dist-test-slave-zkpd krb5kdc[8257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:42:57.125562 8343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788571083ebf4727b4d5c1d3e94ee05a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34427 } }
I20260811 07:42:57.125818 8343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:57.125919 8343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788571083ebf4727b4d5c1d3e94ee05a, State: Initialized, Role: FOLLOWER
I20260811 07:42:57.126382 8343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788571083ebf4727b4d5c1d3e94ee05a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34427 } }
I20260811 07:42:57.126508 8343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:57.126591 8343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:57.126684 8343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:57.127493 8343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788571083ebf4727b4d5c1d3e94ee05a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34427 } }
I20260811 07:42:57.127820 8343 leader_election.cc:302] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 788571083ebf4727b4d5c1d3e94ee05a; no voters:
I20260811 07:42:57.128212 8343 leader_election.cc:288] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:42:57.128279 8349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:57.128511 8349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [term 1 LEADER]: Becoming Leader. State: Replica: 788571083ebf4727b4d5c1d3e94ee05a, State: Running, Role: LEADER
I20260811 07:42:57.128875 8349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788571083ebf4727b4d5c1d3e94ee05a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34427 } }
I20260811 07:42:57.129500 8343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:57.130007 8349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 788571083ebf4727b4d5c1d3e94ee05a. Latest consensus state: current_term: 1 leader_uuid: "788571083ebf4727b4d5c1d3e94ee05a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788571083ebf4727b4d5c1d3e94ee05a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34427 } } }
I20260811 07:42:57.130440 8351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "788571083ebf4727b4d5c1d3e94ee05a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788571083ebf4727b4d5c1d3e94ee05a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34427 } } }
I20260811 07:42:57.130550 8351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:57.130796 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:57.116978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38759 (local address 127.0.0.1:34427)
0811 07:42:57.117420 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:57.117436 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:57.117471 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:42:57.118115 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:57.118133 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:57.118468 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:57.118731 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:57.119667 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:57.120806 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:57.121661 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:57.122147 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:57.122312 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:57.124905 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:57.124910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:57.124923 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:57.124949 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:57.126907 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:57.127406 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:57.127412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:57.127418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:57.127539 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:57.128057 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:57.128060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:57.128062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:57.128592 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:57.129367 (+ 775us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:57.129498 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 07:42:57.129724 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":138,"threads_started":1}
I20260811 07:42:57.130125 8349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:57.134307 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 07:42:57.134737 8366 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:42:57.134812 8366 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:42:57.134873 8359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:57.135751 8359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:57.141170 8359 catalog_manager.cc:1347] Generated new cluster ID: 88a011ab16b443949115be541a8e83cf
I20260811 07:42:57.141223 8359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:57.148048 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:57.136386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57585 (local address 127.0.0.1:34427)
0811 07:42:57.136491 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:57.136495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:57.139643 (+ 3148us) server_negotiation.cc:407] Connection header received
0811 07:42:57.139783 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:57.139786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:57.139832 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:57.139940 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:57.140565 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:57.141233 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:57.142427 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:57.142757 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:57.142846 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:57.144537 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:57.144541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:57.144543 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:42:57.144565 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:57.146201 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:57.146737 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:57.146740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:57.146741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:57.146815 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:42:57.147185 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:42:57.147188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:42:57.147189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:42:57.147400 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:57.147531 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:57.147804 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 07:42:57.147908 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:42:57.154388 8359 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:42:57.155535 8359 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:57.160933 8359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 788571083ebf4727b4d5c1d3e94ee05a: Generated new TSK 0
I20260811 07:42:57.161737 8359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:42:57.169485 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 8271
W20260811 07:42:57.178956 8255 connection.cc:535] client connection to 127.0.0.1:34427 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34427): Connection reset by peer (error 104)
2026-08-11T07:42:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:42:57 dist-test-slave-zkpd krb5kdc[8378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:42:57 dist-test-slave-zkpd krb5kdc[8378](info): set up 2 sockets
Aug 11 07:42:57 dist-test-slave-zkpd krb5kdc[8378](info): commencing operation
krb5kdc: starting...
W20260811 07:42:58.252246 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:42:58 dist-test-slave-zkpd krb5kdc[8378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:42:58.384727 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:58.499697 8392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:58.499965 8392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:58.500007 8392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:58.503963 8392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:58.504014 8392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:58.504033 8392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:58.504050 8392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:58.504065 8392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:58.509092 8392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38069
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:58.509969 8392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:58.510885 8392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:58.519217 8397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:58.519230 8398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:58.519306 8392 server_base.cc:1034] running on GCE node
W20260811 07:42:58.519230 8400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:58.520030 8392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:58.521118 8392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:58.522321 8392 hybrid_clock.cc:648] HybridClock initialized: now 1786434178522308 us; error 26 us; skew 500 ppm
Aug 11 07:42:58 dist-test-slave-zkpd krb5kdc[8378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434178, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:42:58.525117 8392 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:42:58.525944 8392 webserver.cc:460] Webserver started at http://127.0.0.1:38981/ using document root <none> and password file <none>
I20260811 07:42:58.526494 8392 fs_manager.cc:359] Metadata directory not provided
I20260811 07:42:58.526583 8392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:42:58.526815 8392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:42:58.528864 8392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d"
format_stamp: "Formatted at 2026-08-11 07:42:58 on dist-test-slave-zkpd"
server_key: "180058dbb8a3a91708d37abc5e0d281a"
server_key_iv: "aeeede9216e5f8656cf7e363c6dfa0f3"
server_key_version: "encryptionkey@0"
I20260811 07:42:58.529440 8392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d"
format_stamp: "Formatted at 2026-08-11 07:42:58 on dist-test-slave-zkpd"
server_key: "180058dbb8a3a91708d37abc5e0d281a"
server_key_iv: "aeeede9216e5f8656cf7e363c6dfa0f3"
server_key_version: "encryptionkey@0"
I20260811 07:42:58.533034 8392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 07:42:58.535362 8407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:42:58.536605 8392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:42:58.536747 8392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d"
format_stamp: "Formatted at 2026-08-11 07:42:58 on dist-test-slave-zkpd"
server_key: "180058dbb8a3a91708d37abc5e0d281a"
server_key_iv: "aeeede9216e5f8656cf7e363c6dfa0f3"
server_key_version: "encryptionkey@0"
I20260811 07:42:58.536862 8392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:42:58.556011 8392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:42:58.559130 8392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:42:58.559332 8392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:42:58.566363 8392 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38069
I20260811 07:42:58.566425 8462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38069 every 8 connection(s)
I20260811 07:42:58.567276 8392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:42:58.569929 8463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:42:58.572052 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 8392
I20260811 07:42:58.572197 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:42:58.572508 18723 external_mini_cluster.cc:1428] Setting key 322a72f19289833d22f9509674270230
I20260811 07:42:58.576499 8463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d: Bootstrap starting.
I20260811 07:42:58.578634 8463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d: Neither blocks nor log segments found. Creating new log.
I20260811 07:42:58.579368 8463 log.cc:824] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:42:58.581089 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:58.573800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33865 (local address 127.0.0.1:38069)
0811 07:42:58.574233 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:58.574252 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:58.574287 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:42:58.574930 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:58.574948 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:58.575265 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:58.575583 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:58.576049 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:58.577011 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:58.577897 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:58.578353 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:58.578524 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:58.579399 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:58.579404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:58.579417 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:58.579822 (+ 405us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:58.579873 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:58.579938 (+ 65us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:58.580084 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:58.580277 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 07:42:58.580411 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":130,"threads_started":1}
I20260811 07:42:58.581369 8463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d: No bootstrap required, opened a new log
I20260811 07:42:58.583984 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:42:58.585440 8463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38069 } }
I20260811 07:42:58.585665 8463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:42:58.585723 8463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e90963b8fb14d7cb8d13d17ac06ba1d, State: Initialized, Role: FOLLOWER
I20260811 07:42:58.586206 8463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38069 } }
I20260811 07:42:58.586372 8463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:42:58.586455 8463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:42:58.586551 8463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:42:58.587604 8463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38069 } }
I20260811 07:42:58.587989 8463 leader_election.cc:302] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e90963b8fb14d7cb8d13d17ac06ba1d; no voters:
I20260811 07:42:58.588279 8463 leader_election.cc:288] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:42:58.588382 8474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:42:58.588594 8474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [term 1 LEADER]: Becoming Leader. State: Replica: 7e90963b8fb14d7cb8d13d17ac06ba1d, State: Running, Role: LEADER
I20260811 07:42:58.588920 8474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38069 } }
I20260811 07:42:58.589581 8463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:42:58.590755 8476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e90963b8fb14d7cb8d13d17ac06ba1d. Latest consensus state: current_term: 1 leader_uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38069 } } }
I20260811 07:42:58.590880 8476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:58.591192 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:42:58.586236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33145 (local address 127.0.0.1:38069)
0811 07:42:58.586369 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:42:58.586373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:42:58.586386 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:42:58.586418 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:42:58.586420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:42:58.586482 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:42:58.586550 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:42:58.586915 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:58.587624 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:58.588887 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:42:58.589221 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:42:58.589330 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:42:58.590273 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:42:58.590278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:42:58.590283 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:42:58.590300 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:42:58.590329 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:42:58.590343 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:42:58.590445 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:42:58.591017 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 07:42:58.591049 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:42:58.592068 8475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e90963b8fb14d7cb8d13d17ac06ba1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38069 } } }
I20260811 07:42:58.592242 8475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d [sys.catalog]: This master's current role is: LEADER
I20260811 07:42:58.592164 8484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:42:58.594997 8484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:42:58.600319 8484 catalog_manager.cc:1347] Generated new cluster ID: 645a9f7dc2a240909efe441c2e877d4e
I20260811 07:42:58.600386 8484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:42:58.622759 8484 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:42:58.623744 8484 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:42:58.636879 8484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e90963b8fb14d7cb8d13d17ac06ba1d: Generated new TSK 0
I20260811 07:42:58.637697 8484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:42:58.642508 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 8392
W20260811 07:42:58.650830 8376 connection.cc:535] client connection to 127.0.0.1:38069 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38069): Connection reset by peer (error 104)
2026-08-11T07:42:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:42:58 dist-test-slave-zkpd krb5kdc[8499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:42:58 dist-test-slave-zkpd krb5kdc[8499](info): set up 2 sockets
Aug 11 07:42:58 dist-test-slave-zkpd krb5kdc[8499](info): commencing operation
krb5kdc: starting...
W20260811 07:42:59.728546 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:42:59 dist-test-slave-zkpd krb5kdc[8499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:42:59.860497 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35091
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:42:59.973444 8512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:42:59.973677 8512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:42:59.973737 8512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:42:59.977624 8512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:42:59.977674 8512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:42:59.977694 8512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:42:59.977710 8512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:42:59.977725 8512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:42:59.982740 8512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35091
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:42:59.983603 8512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:42:59.984529 8512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:42:59.992654 8518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:42:59.992655 8521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:59.992808 8512 server_base.cc:1034] running on GCE node
W20260811 07:42:59.993003 8519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:42:59.993613 8512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:42:59.994643 8512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:42:59.995800 8512 hybrid_clock.cc:648] HybridClock initialized: now 1786434179995774 us; error 35 us; skew 500 ppm
Aug 11 07:42:59 dist-test-slave-zkpd krb5kdc[8499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:00.002480 8512 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:00.003358 8512 webserver.cc:460] Webserver started at http://127.0.0.1:34287/ using document root <none> and password file <none>
I20260811 07:43:00.003976 8512 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:00.004045 8512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:00.004273 8512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:00.006333 8512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "186306c5ef3d449c9a3f3cb9a8abbb79"
format_stamp: "Formatted at 2026-08-11 07:43:00 on dist-test-slave-zkpd"
server_key: "ff540264eee8c36c96f7ce01ae50a727"
server_key_iv: "9fc0635060accbb3ba24aaeed17e0aae"
server_key_version: "encryptionkey@0"
I20260811 07:43:00.006939 8512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "186306c5ef3d449c9a3f3cb9a8abbb79"
format_stamp: "Formatted at 2026-08-11 07:43:00 on dist-test-slave-zkpd"
server_key: "ff540264eee8c36c96f7ce01ae50a727"
server_key_iv: "9fc0635060accbb3ba24aaeed17e0aae"
server_key_version: "encryptionkey@0"
I20260811 07:43:00.010622 8512 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 07:43:00.013010 8528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:00.014186 8512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:43:00.014333 8512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "186306c5ef3d449c9a3f3cb9a8abbb79"
format_stamp: "Formatted at 2026-08-11 07:43:00 on dist-test-slave-zkpd"
server_key: "ff540264eee8c36c96f7ce01ae50a727"
server_key_iv: "9fc0635060accbb3ba24aaeed17e0aae"
server_key_version: "encryptionkey@0"
I20260811 07:43:00.014432 8512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:00.019106 8512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:00.021916 8512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:00.022104 8512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:00.029238 8512 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35091
I20260811 07:43:00.029251 8582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35091 every 8 connection(s)
I20260811 07:43:00.030125 8512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:00.032961 8583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:00.037446 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 8512
I20260811 07:43:00.037607 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:00.037907 18723 external_mini_cluster.cc:1428] Setting key d57e284ec4c2e946bcdde42b847a8d0d
I20260811 07:43:00.038676 8583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79: Bootstrap starting.
I20260811 07:43:00.041014 8583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:00.041956 8583 log.cc:824] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:00.043449 8586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:00.039354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56977 (local address 127.0.0.1:35091)
0811 07:43:00.039804 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:00.039818 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:00.039854 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:43:00.040543 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:00.040560 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:00.040859 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:00.041208 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:00.042177 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:00.042181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:00.042194 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:00.042205 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:00.042267 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:00.042288 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:00.042424 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:00.042559 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 07:43:00.042721 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":144,"threads_started":1}
I20260811 07:43:00.044035 8583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79: No bootstrap required, opened a new log
I20260811 07:43:00.046475 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:00.048333 8583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186306c5ef3d449c9a3f3cb9a8abbb79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35091 } }
I20260811 07:43:00.048606 8583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:00.048735 8583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186306c5ef3d449c9a3f3cb9a8abbb79, State: Initialized, Role: FOLLOWER
I20260811 07:43:00.049269 8583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186306c5ef3d449c9a3f3cb9a8abbb79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35091 } }
I20260811 07:43:00.049471 8583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:00.049546 8583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:00.049654 8583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:00.050341 8586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:00.048587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51883 (local address 127.0.0.1:35091)
0811 07:43:00.048758 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:00.048762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:00.048890 (+ 128us) server_negotiation.cc:407] Connection header received
0811 07:43:00.049044 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:00.049047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:00.049080 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:00.049156 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:00.049935 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:00.049937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:00.049939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:00.049949 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:00.049973 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:00.049984 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:00.050055 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:00.050162 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 07:43:00.050191 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 07:43:00.050911 8583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186306c5ef3d449c9a3f3cb9a8abbb79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35091 } }
I20260811 07:43:00.051299 8583 leader_election.cc:302] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186306c5ef3d449c9a3f3cb9a8abbb79; no voters:
I20260811 07:43:00.051673 8583 leader_election.cc:288] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:00.051811 8593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:00.052170 8593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [term 1 LEADER]: Becoming Leader. State: Replica: 186306c5ef3d449c9a3f3cb9a8abbb79, State: Running, Role: LEADER
I20260811 07:43:00.052496 8593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186306c5ef3d449c9a3f3cb9a8abbb79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35091 } }
I20260811 07:43:00.053031 8583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:00.054234 8594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "186306c5ef3d449c9a3f3cb9a8abbb79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186306c5ef3d449c9a3f3cb9a8abbb79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35091 } } }
I20260811 07:43:00.054361 8594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:00.054706 8595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186306c5ef3d449c9a3f3cb9a8abbb79. Latest consensus state: current_term: 1 leader_uuid: "186306c5ef3d449c9a3f3cb9a8abbb79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186306c5ef3d449c9a3f3cb9a8abbb79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35091 } } }
I20260811 07:43:00.054806 8595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:00.054785 8602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:00.058084 8602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:00.063208 8602 catalog_manager.cc:1347] Generated new cluster ID: 9aa7f94c58684dcabc9487a16a81d0b7
I20260811 07:43:00.063274 8602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:00.100332 8602 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:00.100708 8602 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:00.105744 8602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 186306c5ef3d449c9a3f3cb9a8abbb79: Generated new TSK 0
I20260811 07:43:00.106483 8602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:00.116185 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 8512
2026-08-11T07:43:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1473 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:00 dist-test-slave-zkpd krb5kdc[8615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:00 dist-test-slave-zkpd krb5kdc[8615](info): set up 2 sockets
Aug 11 07:43:00 dist-test-slave-zkpd krb5kdc[8615](info): commencing operation
krb5kdc: starting...
W20260811 07:43:01.192377 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:01 dist-test-slave-zkpd krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:01.316028 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:01.426891 8631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:01.427132 8631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:01.427211 8631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:01.431195 8631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:01.431247 8631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:01.431285 8631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:01.431322 8631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:01.431355 8631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:01.436478 8631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:01.437455 8631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:01.438522 8631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:43:01.447187 8631 server_base.cc:1034] running on GCE node
W20260811 07:43:01.447083 8638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:01.447132 8640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:01.447083 8637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:01.448000 8631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:01.449077 8631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:01.450238 8631 hybrid_clock.cc:648] HybridClock initialized: now 1786434181450212 us; error 43 us; skew 500 ppm
Aug 11 07:43:01 dist-test-slave-zkpd krb5kdc[8615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434181, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:01.452888 8631 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:01.453758 8631 webserver.cc:460] Webserver started at http://127.0.0.1:45307/ using document root <none> and password file <none>
I20260811 07:43:01.454308 8631 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:01.454396 8631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:01.454625 8631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:01.456689 8631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "a3b1b51c2537479c936c1dcb5736971c"
format_stamp: "Formatted at 2026-08-11 07:43:01 on dist-test-slave-zkpd"
server_key: "3b117d164371ac3a394306af2493da1b"
server_key_iv: "4d5db3748790990207a32263ccfaea62"
server_key_version: "encryptionkey@0"
I20260811 07:43:01.457255 8631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "a3b1b51c2537479c936c1dcb5736971c"
format_stamp: "Formatted at 2026-08-11 07:43:01 on dist-test-slave-zkpd"
server_key: "3b117d164371ac3a394306af2493da1b"
server_key_iv: "4d5db3748790990207a32263ccfaea62"
server_key_version: "encryptionkey@0"
I20260811 07:43:01.461326 8631 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 07:43:01.464169 8649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:01.465583 8631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:43:01.465730 8631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "a3b1b51c2537479c936c1dcb5736971c"
format_stamp: "Formatted at 2026-08-11 07:43:01 on dist-test-slave-zkpd"
server_key: "3b117d164371ac3a394306af2493da1b"
server_key_iv: "4d5db3748790990207a32263ccfaea62"
server_key_version: "encryptionkey@0"
I20260811 07:43:01.465845 8631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:01.487160 8631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:01.489975 8631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:01.490173 8631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:01.497426 8631 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41057
I20260811 07:43:01.497524 8702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41057 every 8 connection(s)
I20260811 07:43:01.498579 8631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:01.501674 8703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:01.501989 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 8631
I20260811 07:43:01.502146 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:01.502452 18723 external_mini_cluster.cc:1428] Setting key 113b573c695b861013692c850eb9f031
I20260811 07:43:01.508997 8703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c: Bootstrap starting.
I20260811 07:43:01.511682 8703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c: Neither blocks nor log segments found. Creating new log.
Aug 11 07:43:01 dist-test-slave-zkpd krb5kdc[8615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:43:01.512638 8703 log.cc:824] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:01.514725 8703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c: No bootstrap required, opened a new log
I20260811 07:43:01.517225 8706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:01.503881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46485 (local address 127.0.0.1:41057)
0811 07:43:01.504351 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:01.504375 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:01.504419 (+ 44us) server_negotiation.cc:407] Connection header received
0811 07:43:01.505178 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:01.505203 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:01.505656 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:01.506039 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:01.506691 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:01.508257 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:01.509101 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:01.509780 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:01.509945 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:01.512685 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:01.512690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:01.512702 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:01.512751 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:01.514451 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:01.514961 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:01.514966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:01.514971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:01.515074 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:01.515377 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:01.515379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:01.515380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:01.515726 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:01.515867 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:01.516294 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 07:43:01.516569 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260811 07:43:01.518568 8703 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b1b51c2537479c936c1dcb5736971c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41057 } }
I20260811 07:43:01.518819 8703 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:01.518869 8703 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3b1b51c2537479c936c1dcb5736971c, State: Initialized, Role: FOLLOWER
I20260811 07:43:01.519371 8703 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b1b51c2537479c936c1dcb5736971c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41057 } }
I20260811 07:43:01.519551 8703 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:01.519631 8703 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:01.519733 8703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:01.520133 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:01.520710 8703 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b1b51c2537479c936c1dcb5736971c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41057 } }
I20260811 07:43:01.521034 8703 leader_election.cc:302] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3b1b51c2537479c936c1dcb5736971c; no voters:
I20260811 07:43:01.521356 8703 leader_election.cc:288] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:01.521492 8712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:01.521705 8712 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [term 1 LEADER]: Becoming Leader. State: Replica: a3b1b51c2537479c936c1dcb5736971c, State: Running, Role: LEADER
I20260811 07:43:01.522073 8712 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b1b51c2537479c936c1dcb5736971c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41057 } }
I20260811 07:43:01.522780 8703 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:01.523350 8716 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3b1b51c2537479c936c1dcb5736971c. Latest consensus state: current_term: 1 leader_uuid: "a3b1b51c2537479c936c1dcb5736971c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b1b51c2537479c936c1dcb5736971c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41057 } } }
I20260811 07:43:01.523490 8716 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:01.523754 8715 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3b1b51c2537479c936c1dcb5736971c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b1b51c2537479c936c1dcb5736971c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41057 } } }
I20260811 07:43:01.523926 8715 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c [sys.catalog]: This master's current role is: LEADER
W20260811 07:43:01.527995 8731 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:43:01.528110 8731 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:43:01.528450 8721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:01.529331 8721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:01.535125 8706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:01.522498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40911 (local address 127.0.0.1:41057)
0811 07:43:01.522591 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:01.522595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:01.522608 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:43:01.522641 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:01.522643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:01.522681 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:01.522761 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:01.524457 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:01.525216 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:01.526129 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:01.526451 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:01.526553 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:01.528156 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:01.528160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:01.528162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:01.528190 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:01.532944 (+ 4754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:01.533443 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:01.533447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:01.533449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:01.533536 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:01.533883 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:01.533889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:01.533890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:01.534104 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:01.534229 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:01.534856 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 07:43:01.534976 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 07:43:01.535671 8721 catalog_manager.cc:1347] Generated new cluster ID: 07972b2d05b4453ab623e085c8b3db2f
I20260811 07:43:01.535732 8721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:01.561769 8721 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:01.562911 8721 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:01.569123 8721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3b1b51c2537479c936c1dcb5736971c: Generated new TSK 0
I20260811 07:43:01.569860 8721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:01.575637 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 8631
2026-08-11T07:43:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:01 dist-test-slave-zkpd krb5kdc[8737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:01 dist-test-slave-zkpd krb5kdc[8737](info): set up 2 sockets
Aug 11 07:43:01 dist-test-slave-zkpd krb5kdc[8737](info): commencing operation
krb5kdc: starting...
W20260811 07:43:02.640486 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:02 dist-test-slave-zkpd krb5kdc[8737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:02.762779 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:02.873152 8750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:02.873374 8750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:02.873414 8750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:02.877229 8750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:02.877274 8750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:02.877293 8750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:02.877310 8750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:02.877324 8750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:02.882225 8750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:02.883087 8750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:02.884008 8750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:02.892097 8760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:02.892124 8756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:02.892098 8757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:02.892232 8750 server_base.cc:1034] running on GCE node
I20260811 07:43:02.892868 8750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:02.893857 8750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:02.895047 8750 hybrid_clock.cc:648] HybridClock initialized: now 1786434182895015 us; error 42 us; skew 500 ppm
Aug 11 07:43:02 dist-test-slave-zkpd krb5kdc[8737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434182, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:02.897846 8750 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:02.898720 8750 webserver.cc:460] Webserver started at http://127.0.0.1:37707/ using document root <none> and password file <none>
I20260811 07:43:02.899258 8750 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:02.899371 8750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:02.899593 8750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:02.901619 8750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "44c366176cf44c49ba34b43a13eddf97"
format_stamp: "Formatted at 2026-08-11 07:43:02 on dist-test-slave-zkpd"
server_key: "37317d3c99072bca5caf3aa9bb6514f5"
server_key_iv: "02c90cdfdd8700c5a43eede14418f210"
server_key_version: "encryptionkey@0"
I20260811 07:43:02.902184 8750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "44c366176cf44c49ba34b43a13eddf97"
format_stamp: "Formatted at 2026-08-11 07:43:02 on dist-test-slave-zkpd"
server_key: "37317d3c99072bca5caf3aa9bb6514f5"
server_key_iv: "02c90cdfdd8700c5a43eede14418f210"
server_key_version: "encryptionkey@0"
I20260811 07:43:02.905719 8750 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:43:02.907976 8767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:02.909132 8750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:43:02.909272 8750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "44c366176cf44c49ba34b43a13eddf97"
format_stamp: "Formatted at 2026-08-11 07:43:02 on dist-test-slave-zkpd"
server_key: "37317d3c99072bca5caf3aa9bb6514f5"
server_key_iv: "02c90cdfdd8700c5a43eede14418f210"
server_key_version: "encryptionkey@0"
I20260811 07:43:02.909389 8750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:02.941188 8750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:02.944581 8750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:02.944787 8750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:02.952107 8750 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36225
I20260811 07:43:02.952123 8822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36225 every 8 connection(s)
I20260811 07:43:02.953090 8750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:02.955876 8823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:02.959014 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 8750
I20260811 07:43:02.959158 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:02.959503 18723 external_mini_cluster.cc:1428] Setting key 1d1b5716b32d01e076851083914f3edf
I20260811 07:43:02.961906 8823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97: Bootstrap starting.
I20260811 07:43:02.964316 8823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:02.965200 8823 log.cc:824] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:02.967311 8823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97: No bootstrap required, opened a new log
I20260811 07:43:02.968297 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:02.960852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49819 (local address 127.0.0.1:36225)
0811 07:43:02.961253 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:02.961268 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:02.961302 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:43:02.961883 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:02.961898 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:02.962264 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:02.962528 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:02.962831 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:02.963842 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:02.965105 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:02.965692 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:02.965861 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:02.966612 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:02.966617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:02.966630 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:02.966987 (+ 357us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:02.967041 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:02.967063 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:02.967140 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:02.967376 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 07:43:02.967564 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":126,"threads_started":1}
I20260811 07:43:02.971011 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:02.971101 8823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c366176cf44c49ba34b43a13eddf97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } }
I20260811 07:43:02.971356 8823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:02.971450 8823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c366176cf44c49ba34b43a13eddf97, State: Initialized, Role: FOLLOWER
I20260811 07:43:02.971824 8823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c366176cf44c49ba34b43a13eddf97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } }
I20260811 07:43:02.972012 8823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:02.972100 8823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:02.972193 8823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:02.973066 8823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c366176cf44c49ba34b43a13eddf97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } }
I20260811 07:43:02.973402 8823 leader_election.cc:302] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44c366176cf44c49ba34b43a13eddf97; no voters:
I20260811 07:43:02.973737 8823 leader_election.cc:288] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:02.974082 8834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:02.974321 8834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [term 1 LEADER]: Becoming Leader. State: Replica: 44c366176cf44c49ba34b43a13eddf97, State: Running, Role: LEADER
I20260811 07:43:02.974665 8834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c366176cf44c49ba34b43a13eddf97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } }
I20260811 07:43:02.974994 8823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:02.976353 8835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c366176cf44c49ba34b43a13eddf97. Latest consensus state: current_term: 1 leader_uuid: "44c366176cf44c49ba34b43a13eddf97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c366176cf44c49ba34b43a13eddf97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } } }
I20260811 07:43:02.976497 8835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:02.976963 8833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44c366176cf44c49ba34b43a13eddf97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c366176cf44c49ba34b43a13eddf97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } } }
I20260811 07:43:02.977365 8833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:02.977386 8846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:02.977615 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:02.973150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45697 (local address 127.0.0.1:36225)
0811 07:43:02.973252 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:02.973258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:02.973426 (+ 168us) server_negotiation.cc:407] Connection header received
0811 07:43:02.973595 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:02.973598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:02.973643 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:02.973734 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:02.974011 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:02.974659 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:02.975552 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:02.975891 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:02.976451 (+ 560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:02.976967 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:02.976974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:02.976978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:02.976990 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:02.977018 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:02.977035 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:02.977424 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:02.977460 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 07:43:02.977505 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:43:02.980415 8846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:02.986272 8846 catalog_manager.cc:1347] Generated new cluster ID: c39834e8e0de49cd9d751d3227107ae8
I20260811 07:43:02.986399 8846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:03.008201 8846 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:03.009203 8846 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:03.018306 8846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44c366176cf44c49ba34b43a13eddf97: Generated new TSK 0
I20260811 07:43:03.019006 8846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:03.026679 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 8750
2026-08-11T07:43:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:03 dist-test-slave-zkpd krb5kdc[8855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:03 dist-test-slave-zkpd krb5kdc[8855](info): set up 2 sockets
Aug 11 07:43:03 dist-test-slave-zkpd krb5kdc[8855](info): commencing operation
krb5kdc: starting...
W20260811 07:43:04.127923 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:04 dist-test-slave-zkpd krb5kdc[8855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:04.253214 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:04.363668 8869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:04.364009 8869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:04.364092 8869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:04.368022 8869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:04.368112 8869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:04.368150 8869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:04.368209 8869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:04.368248 8869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:04.373396 8869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:04.374547 8869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:04.375550 8869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:04.384120 8875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:04.384106 8877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:04.384217 8869 server_base.cc:1034] running on GCE node
W20260811 07:43:04.384106 8874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:04.384963 8869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:04.385986 8869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:04.387173 8869 hybrid_clock.cc:648] HybridClock initialized: now 1786434184387157 us; error 32 us; skew 500 ppm
Aug 11 07:43:04 dist-test-slave-zkpd krb5kdc[8855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434184, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:04.389936 8869 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:04.390810 8869 webserver.cc:460] Webserver started at http://127.0.0.1:35923/ using document root <none> and password file <none>
I20260811 07:43:04.391359 8869 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:04.391456 8869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:04.391686 8869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:04.393720 8869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "a6f38d22926e4fc1b22dcc9a9940aeda"
format_stamp: "Formatted at 2026-08-11 07:43:04 on dist-test-slave-zkpd"
server_key: "d242e65ed3e85a70f0faa4134e268f83"
server_key_iv: "915883ed448e07b427ac60046a26f39d"
server_key_version: "encryptionkey@0"
I20260811 07:43:04.394284 8869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "a6f38d22926e4fc1b22dcc9a9940aeda"
format_stamp: "Formatted at 2026-08-11 07:43:04 on dist-test-slave-zkpd"
server_key: "d242e65ed3e85a70f0faa4134e268f83"
server_key_iv: "915883ed448e07b427ac60046a26f39d"
server_key_version: "encryptionkey@0"
I20260811 07:43:04.397887 8869 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:43:04.400198 8884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:04.401369 8869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 07:43:04.401527 8869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "a6f38d22926e4fc1b22dcc9a9940aeda"
format_stamp: "Formatted at 2026-08-11 07:43:04 on dist-test-slave-zkpd"
server_key: "d242e65ed3e85a70f0faa4134e268f83"
server_key_iv: "915883ed448e07b427ac60046a26f39d"
server_key_version: "encryptionkey@0"
I20260811 07:43:04.401643 8869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:04.406157 8869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:04.409348 8869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:04.409543 8869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:04.416659 8869 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34843
I20260811 07:43:04.416699 8943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34843 every 8 connection(s)
I20260811 07:43:04.417542 8869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:04.418906 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 8869
I20260811 07:43:04.419056 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:04.419420 18723 external_mini_cluster.cc:1428] Setting key f868cc74f9c2705adad08e39640ca5a9
I20260811 07:43:04.420866 8944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:04.424790 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:04.420691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38879 (local address 127.0.0.1:34843)
0811 07:43:04.421136 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:04.421154 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:04.421194 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:43:04.422035 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:04.422058 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:04.422419 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:04.422763 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:04.423419 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:04.423437 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:04.423452 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:04.423462 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:04.423531 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:04.423556 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:04.423690 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:04.423754 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 07:43:04.423916 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":164,"threads_started":1}
I20260811 07:43:04.427619 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:04.427764 8944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda: Bootstrap starting.
I20260811 07:43:04.430572 8944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:04.431483 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:04.429864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55265 (local address 127.0.0.1:34843)
0811 07:43:04.429981 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:04.429985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:04.430229 (+ 244us) server_negotiation.cc:407] Connection header received
0811 07:43:04.430341 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:04.430353 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:04.430381 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:04.430443 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:04.431086 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:04.431088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:04.431090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:04.431100 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:04.431122 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:04.431132 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:04.431197 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:04.431315 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 07:43:04.431355 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:43:04.431548 8944 log.cc:824] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:04.433378 8944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda: No bootstrap required, opened a new log
I20260811 07:43:04.436729 8944 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f38d22926e4fc1b22dcc9a9940aeda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34843 } }
I20260811 07:43:04.436940 8944 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:04.437073 8944 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f38d22926e4fc1b22dcc9a9940aeda, State: Initialized, Role: FOLLOWER
I20260811 07:43:04.437546 8944 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f38d22926e4fc1b22dcc9a9940aeda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34843 } }
I20260811 07:43:04.437701 8944 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:04.437803 8944 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:04.437907 8944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:04.438755 8944 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f38d22926e4fc1b22dcc9a9940aeda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34843 } }
I20260811 07:43:04.439101 8944 leader_election.cc:302] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6f38d22926e4fc1b22dcc9a9940aeda; no voters:
I20260811 07:43:04.439399 8944 leader_election.cc:288] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:04.439536 8955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:04.439791 8955 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [term 1 LEADER]: Becoming Leader. State: Replica: a6f38d22926e4fc1b22dcc9a9940aeda, State: Running, Role: LEADER
I20260811 07:43:04.440197 8955 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f38d22926e4fc1b22dcc9a9940aeda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34843 } }
I20260811 07:43:04.440658 8944 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:04.441987 8957 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6f38d22926e4fc1b22dcc9a9940aeda. Latest consensus state: current_term: 1 leader_uuid: "a6f38d22926e4fc1b22dcc9a9940aeda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f38d22926e4fc1b22dcc9a9940aeda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34843 } } }
I20260811 07:43:04.442016 8956 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6f38d22926e4fc1b22dcc9a9940aeda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f38d22926e4fc1b22dcc9a9940aeda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34843 } } }
I20260811 07:43:04.442165 8956 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:04.442422 8957 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:04.443187 8966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:04.445940 8966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:04.451833 8966 catalog_manager.cc:1347] Generated new cluster ID: f90cefc66baf48c4a69292eb8316fe63
I20260811 07:43:04.451936 8966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:04.461733 8966 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:04.462033 8966 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:04.466269 8966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6f38d22926e4fc1b22dcc9a9940aeda: Generated new TSK 0
I20260811 07:43:04.466877 8966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:04.475090 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 8869
2026-08-11T07:43:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:04 dist-test-slave-zkpd krb5kdc[8976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:04 dist-test-slave-zkpd krb5kdc[8976](info): set up 2 sockets
Aug 11 07:43:04 dist-test-slave-zkpd krb5kdc[8976](info): commencing operation
krb5kdc: starting...
W20260811 07:43:05.549372 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:05 dist-test-slave-zkpd krb5kdc[8976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:05.675021 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:05.786387 8992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:05.786667 8992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:05.786769 8992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:05.790632 8992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:05.790725 8992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:05.790764 8992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:05.790823 8992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:05.790864 8992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:05.795912 8992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:05.796941 8992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:05.797915 8992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:05.806576 8998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:05.806607 8997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:05.806576 9000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:05.806810 8992 server_base.cc:1034] running on GCE node
I20260811 07:43:05.807418 8992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:05.808506 8992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:05.809703 8992 hybrid_clock.cc:648] HybridClock initialized: now 1786434185809682 us; error 31 us; skew 500 ppm
Aug 11 07:43:05 dist-test-slave-zkpd krb5kdc[8976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434185, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:05.816902 8992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:05.817771 8992 webserver.cc:460] Webserver started at http://127.0.0.1:41091/ using document root <none> and password file <none>
I20260811 07:43:05.818356 8992 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:05.818452 8992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:05.818686 8992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:05.820856 8992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "e3e69af44e6b40dc8ec1797084b6c6d7"
format_stamp: "Formatted at 2026-08-11 07:43:05 on dist-test-slave-zkpd"
server_key: "81ee408e3bf626d218100bbe4dd14ba8"
server_key_iv: "0d609b324dfd838fc15a46c4068bbb31"
server_key_version: "encryptionkey@0"
I20260811 07:43:05.821440 8992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "e3e69af44e6b40dc8ec1797084b6c6d7"
format_stamp: "Formatted at 2026-08-11 07:43:05 on dist-test-slave-zkpd"
server_key: "81ee408e3bf626d218100bbe4dd14ba8"
server_key_iv: "0d609b324dfd838fc15a46c4068bbb31"
server_key_version: "encryptionkey@0"
I20260811 07:43:05.825122 8992 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:43:05.827479 9009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:05.828740 8992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:05.828896 8992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "e3e69af44e6b40dc8ec1797084b6c6d7"
format_stamp: "Formatted at 2026-08-11 07:43:05 on dist-test-slave-zkpd"
server_key: "81ee408e3bf626d218100bbe4dd14ba8"
server_key_iv: "0d609b324dfd838fc15a46c4068bbb31"
server_key_version: "encryptionkey@0"
I20260811 07:43:05.829008 8992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:05.847489 8992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:05.850376 8992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:05.850584 8992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:05.857508 8992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43231
I20260811 07:43:05.857533 9064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43231 every 8 connection(s)
I20260811 07:43:05.858426 8992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:05.861142 9067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:05.861713 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 8992
I20260811 07:43:05.861877 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:05.862205 18723 external_mini_cluster.cc:1428] Setting key abc46aa411dc0cf8323a219467fb6182
I20260811 07:43:05.867690 9067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7: Bootstrap starting.
I20260811 07:43:05.869943 9067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:05.870769 9067 log.cc:824] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7: Log is configured to *not* fsync() on all Append() calls
Aug 11 07:43:05 dist-test-slave-zkpd krb5kdc[8976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:43:05.872840 9067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7: No bootstrap required, opened a new log
I20260811 07:43:05.876073 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:05.863643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:60345 (local address 127.0.0.1:43231)
0811 07:43:05.864098 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:05.864112 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:05.864145 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:43:05.864795 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:05.864811 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:05.865132 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:05.865464 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:05.866295 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:05.867269 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:05.868257 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:05.868700 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:05.868877 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:05.871515 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:05.871522 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:05.871544 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:05.871568 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:05.873211 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:05.873763 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:05.873767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:05.873771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:05.873872 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:05.874201 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:05.874203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:05.874204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:05.874555 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:05.874702 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:05.875077 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 07:43:05.875282 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":144,"threads_started":1}
I20260811 07:43:05.876920 9067 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e69af44e6b40dc8ec1797084b6c6d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43231 } }
I20260811 07:43:05.877200 9067 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:05.877260 9067 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e69af44e6b40dc8ec1797084b6c6d7, State: Initialized, Role: FOLLOWER
I20260811 07:43:05.877696 9067 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e69af44e6b40dc8ec1797084b6c6d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43231 } }
I20260811 07:43:05.877861 9067 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:05.877956 9067 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:05.878062 9067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:05.879009 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:05.879084 9067 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e69af44e6b40dc8ec1797084b6c6d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43231 } }
I20260811 07:43:05.879438 9067 leader_election.cc:302] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3e69af44e6b40dc8ec1797084b6c6d7; no voters:
I20260811 07:43:05.879755 9067 leader_election.cc:288] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:05.879830 9072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:05.880043 9072 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [term 1 LEADER]: Becoming Leader. State: Replica: e3e69af44e6b40dc8ec1797084b6c6d7, State: Running, Role: LEADER
I20260811 07:43:05.880409 9072 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e69af44e6b40dc8ec1797084b6c6d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43231 } }
I20260811 07:43:05.881026 9067 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:05.881599 9072 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3e69af44e6b40dc8ec1797084b6c6d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e69af44e6b40dc8ec1797084b6c6d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43231 } } }
I20260811 07:43:05.881788 9072 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:05.882074 9073 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3e69af44e6b40dc8ec1797084b6c6d7. Latest consensus state: current_term: 1 leader_uuid: "e3e69af44e6b40dc8ec1797084b6c6d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3e69af44e6b40dc8ec1797084b6c6d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43231 } } }
I20260811 07:43:05.882200 9073 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:05.882236 9086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:05.885637 9086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:05.892522 9086 catalog_manager.cc:1347] Generated new cluster ID: a1dbecb067fa4ea4ba9db0b8e8941551
I20260811 07:43:05.892578 9086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:05.893256 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:05.883623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:37601 (local address 127.0.0.1:43231)
0811 07:43:05.883712 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:05.883716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:05.884085 (+ 369us) server_negotiation.cc:407] Connection header received
0811 07:43:05.884261 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:05.884265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:05.884314 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:05.884405 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:05.885216 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:05.885912 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:05.887016 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:05.887375 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:05.887488 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:05.889161 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:05.889165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:05.889169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:05.889200 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:05.890946 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:05.891511 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:05.891515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:05.891516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:05.891604 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:05.891995 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:05.891998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:05.892000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:05.892203 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:05.892343 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:05.893000 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 07:43:05.893103 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 07:43:05.900489 9086 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:05.901554 9086 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:05.916913 9086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3e69af44e6b40dc8ec1797084b6c6d7: Generated new TSK 0
I20260811 07:43:05.917608 9086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:05.928716 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 8992
W20260811 07:43:05.937369 8974 connection.cc:535] client connection to 127.0.0.1:43231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43231): Connection reset by peer (error 104)
2026-08-11T07:43:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:05 dist-test-slave-zkpd krb5kdc[9101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:05 dist-test-slave-zkpd krb5kdc[9101](info): set up 2 sockets
Aug 11 07:43:05 dist-test-slave-zkpd krb5kdc[9101](info): commencing operation
krb5kdc: starting...
W20260811 07:43:07.008633 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:07 dist-test-slave-zkpd krb5kdc[9101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:07.136960 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:07.248688 9115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:07.248978 9115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:07.249050 9115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:07.252981 9115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:07.253060 9115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:07.253096 9115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:07.253153 9115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:07.253194 9115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:07.258262 9115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:07.259250 9115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:07.260332 9115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:07.268656 9120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:07.268643 9121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:07.268661 9123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:07.269011 9115 server_base.cc:1034] running on GCE node
I20260811 07:43:07.269546 9115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:07.270545 9115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:07.271706 9115 hybrid_clock.cc:648] HybridClock initialized: now 1786434187271677 us; error 41 us; skew 500 ppm
Aug 11 07:43:07 dist-test-slave-zkpd krb5kdc[9101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434187, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:07.274566 9115 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:07.275395 9115 webserver.cc:460] Webserver started at http://127.0.0.1:37533/ using document root <none> and password file <none>
I20260811 07:43:07.276007 9115 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:07.276072 9115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:07.276300 9115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:07.278318 9115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "70d3f7e4b4f741a88b6036dae5d36cd9"
format_stamp: "Formatted at 2026-08-11 07:43:07 on dist-test-slave-zkpd"
server_key: "e3661c46232d3054fccfd1cafbf90139"
server_key_iv: "9dd1aab14cfcaa0214aca6dfccdc2ae5"
server_key_version: "encryptionkey@0"
I20260811 07:43:07.278873 9115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "70d3f7e4b4f741a88b6036dae5d36cd9"
format_stamp: "Formatted at 2026-08-11 07:43:07 on dist-test-slave-zkpd"
server_key: "e3661c46232d3054fccfd1cafbf90139"
server_key_iv: "9dd1aab14cfcaa0214aca6dfccdc2ae5"
server_key_version: "encryptionkey@0"
I20260811 07:43:07.282699 9115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:43:07.285089 9131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:07.286325 9115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:43:07.286466 9115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "70d3f7e4b4f741a88b6036dae5d36cd9"
format_stamp: "Formatted at 2026-08-11 07:43:07 on dist-test-slave-zkpd"
server_key: "e3661c46232d3054fccfd1cafbf90139"
server_key_iv: "9dd1aab14cfcaa0214aca6dfccdc2ae5"
server_key_version: "encryptionkey@0"
I20260811 07:43:07.286574 9115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:07.305656 9115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:07.309386 9115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:07.309597 9115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:07.316720 9115 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34075
I20260811 07:43:07.316787 9188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34075 every 8 connection(s)
I20260811 07:43:07.317687 9115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:07.320524 9189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:07.323356 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 9115
I20260811 07:43:07.323513 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:07.323819 18723 external_mini_cluster.cc:1428] Setting key c94c366c09071a7ed6e5fbe0d1d32b13
I20260811 07:43:07.326725 9189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9: Bootstrap starting.
I20260811 07:43:07.329398 9189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:07.330135 9189 log.cc:824] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:07.332259 9189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9: No bootstrap required, opened a new log
I20260811 07:43:07.333832 9192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:07.325246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:52163 (local address 127.0.0.1:34075)
0811 07:43:07.325679 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:07.325700 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:07.325742 (+ 42us) server_negotiation.cc:407] Connection header received
0811 07:43:07.326612 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:07.326636 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:07.327068 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:07.327427 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:07.327838 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:07.329355 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:07.330266 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:07.330855 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:07.331027 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:07.331863 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:07.331870 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:07.331887 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:07.332403 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:07.332469 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:07.332498 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:07.332638 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:07.332802 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 07:43:07.332934 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":121,"threads_started":1}
I20260811 07:43:07.336637 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:07.336643 9189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d3f7e4b4f741a88b6036dae5d36cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34075 } }
I20260811 07:43:07.336892 9189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:07.336967 9189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70d3f7e4b4f741a88b6036dae5d36cd9, State: Initialized, Role: FOLLOWER
I20260811 07:43:07.337358 9189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d3f7e4b4f741a88b6036dae5d36cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34075 } }
I20260811 07:43:07.337472 9189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:07.337555 9189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:07.337666 9189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:07.338653 9189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d3f7e4b4f741a88b6036dae5d36cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34075 } }
I20260811 07:43:07.339030 9189 leader_election.cc:302] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70d3f7e4b4f741a88b6036dae5d36cd9; no voters:
I20260811 07:43:07.339382 9189 leader_election.cc:288] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:07.339460 9200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:07.339668 9200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [term 1 LEADER]: Becoming Leader. State: Replica: 70d3f7e4b4f741a88b6036dae5d36cd9, State: Running, Role: LEADER
I20260811 07:43:07.340042 9200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d3f7e4b4f741a88b6036dae5d36cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34075 } }
I20260811 07:43:07.340665 9189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:07.341684 9201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70d3f7e4b4f741a88b6036dae5d36cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d3f7e4b4f741a88b6036dae5d36cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34075 } } }
I20260811 07:43:07.341822 9201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:07.342118 9202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70d3f7e4b4f741a88b6036dae5d36cd9. Latest consensus state: current_term: 1 leader_uuid: "70d3f7e4b4f741a88b6036dae5d36cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d3f7e4b4f741a88b6036dae5d36cd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34075 } } }
I20260811 07:43:07.342195 9202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:07.343073 9212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:07.343436 9192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:07.338856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:40323 (local address 127.0.0.1:34075)
0811 07:43:07.338954 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:07.338957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:07.339142 (+ 185us) server_negotiation.cc:407] Connection header received
0811 07:43:07.339276 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:07.339279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:07.339338 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:07.339420 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:07.339773 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:07.340439 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:07.341434 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:07.341792 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:07.341870 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:07.342857 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:07.342861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:07.342863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:07.342875 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:07.342902 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:07.342917 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:07.343010 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:07.343244 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 07:43:07.343278 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 07:43:07.345839 9212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:07.350951 9212 catalog_manager.cc:1347] Generated new cluster ID: 44d1dfd17e564c629cfd28a9d3ba5f0c
I20260811 07:43:07.351024 9212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:07.373095 9212 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:07.374042 9212 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:07.383266 9212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70d3f7e4b4f741a88b6036dae5d36cd9: Generated new TSK 0
I20260811 07:43:07.383948 9212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:07.394956 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 9115
W20260811 07:43:07.403837 9099 connection.cc:535] client connection to 127.0.0.1:34075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34075): Connection reset by peer (error 104)
2026-08-11T07:43:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:07 dist-test-slave-zkpd krb5kdc[9225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:07 dist-test-slave-zkpd krb5kdc[9225](info): set up 2 sockets
Aug 11 07:43:07 dist-test-slave-zkpd krb5kdc[9225](info): commencing operation
krb5kdc: starting...
W20260811 07:43:08.540499 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:08 dist-test-slave-zkpd krb5kdc[9225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:08.663161 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:08.774190 9238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:08.774434 9238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:08.774475 9238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:08.778299 9238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:08.778353 9238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:08.778414 9238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:08.778439 9238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:08.778478 9238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:08.783547 9238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:08.784571 9238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:08.785492 9238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:08.793882 9243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:08.793928 9244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:08.793928 9246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:08.794171 9238 server_base.cc:1034] running on GCE node
I20260811 07:43:08.794694 9238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:08.795670 9238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:08.796860 9238 hybrid_clock.cc:648] HybridClock initialized: now 1786434188796812 us; error 52 us; skew 500 ppm
Aug 11 07:43:08 dist-test-slave-zkpd krb5kdc[9225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434188, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:08.799466 9238 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:08.800386 9238 webserver.cc:460] Webserver started at http://127.0.0.1:36047/ using document root <none> and password file <none>
I20260811 07:43:08.800933 9238 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:08.801024 9238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:08.801249 9238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:08.803215 9238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "d09a6c6a726b4d6994e2826b177a5582"
format_stamp: "Formatted at 2026-08-11 07:43:08 on dist-test-slave-zkpd"
server_key: "486f001cffc962911623527e4136da66"
server_key_iv: "14265e681ac5da3fada7f0280167a49b"
server_key_version: "encryptionkey@0"
I20260811 07:43:08.803757 9238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "d09a6c6a726b4d6994e2826b177a5582"
format_stamp: "Formatted at 2026-08-11 07:43:08 on dist-test-slave-zkpd"
server_key: "486f001cffc962911623527e4136da66"
server_key_iv: "14265e681ac5da3fada7f0280167a49b"
server_key_version: "encryptionkey@0"
I20260811 07:43:08.807380 9238 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:43:08.809675 9254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:08.810794 9238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:43:08.810940 9238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "d09a6c6a726b4d6994e2826b177a5582"
format_stamp: "Formatted at 2026-08-11 07:43:08 on dist-test-slave-zkpd"
server_key: "486f001cffc962911623527e4136da66"
server_key_iv: "14265e681ac5da3fada7f0280167a49b"
server_key_version: "encryptionkey@0"
I20260811 07:43:08.811043 9238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:08.815451 9238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:08.818172 9238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:08.818361 9238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:08.825364 9238 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44807
I20260811 07:43:08.825393 9309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44807 every 8 connection(s)
I20260811 07:43:08.826234 9238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:08.828827 9310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:08.829262 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 9238
I20260811 07:43:08.829407 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:08.829707 18723 external_mini_cluster.cc:1428] Setting key 62452a36d5e348bb3c0978546b1cf04c
I20260811 07:43:08.835129 9313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:08.831067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:57137 (local address 127.0.0.1:44807)
0811 07:43:08.831478 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:08.831499 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:08.831542 (+ 43us) server_negotiation.cc:407] Connection header received
0811 07:43:08.832362 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:08.832386 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:08.832752 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:08.833117 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:08.833793 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:08.833798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:08.833813 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:08.833824 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:08.833885 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:08.833911 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:08.834048 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:08.834121 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 07:43:08.834269 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
I20260811 07:43:08.836042 9310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582: Bootstrap starting.
I20260811 07:43:08.837818 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:08.838447 9310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:08.839241 9310 log.cc:824] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:08.841243 9310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582: No bootstrap required, opened a new log
I20260811 07:43:08.841509 9313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:08.839877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:37759 (local address 127.0.0.1:44807)
0811 07:43:08.840027 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:08.840031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:08.840270 (+ 239us) server_negotiation.cc:407] Connection header received
0811 07:43:08.840342 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:08.840345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:08.840376 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:08.840463 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:08.841133 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:08.841137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:08.841139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:08.841151 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:08.841176 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:08.841189 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:08.841282 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:08.841369 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 07:43:08.841412 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 07:43:08.845135 9310 raft_consensus.cc:348] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09a6c6a726b4d6994e2826b177a5582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260811 07:43:08.845355 9310 raft_consensus.cc:374] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:08.845389 9310 raft_consensus.cc:729] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d09a6c6a726b4d6994e2826b177a5582, State: Initialized, Role: FOLLOWER
I20260811 07:43:08.845728 9310 consensus_queue.cc:260] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09a6c6a726b4d6994e2826b177a5582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260811 07:43:08.845834 9310 raft_consensus.cc:388] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:08.845880 9310 raft_consensus.cc:482] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:08.845953 9310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:08.846733 9310 raft_consensus.cc:504] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09a6c6a726b4d6994e2826b177a5582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260811 07:43:08.847024 9310 leader_election.cc:302] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d09a6c6a726b4d6994e2826b177a5582; no voters:
I20260811 07:43:08.847252 9310 leader_election.cc:288] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:08.847414 9320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:08.847865 9320 raft_consensus.cc:686] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [term 1 LEADER]: Becoming Leader. State: Replica: d09a6c6a726b4d6994e2826b177a5582, State: Running, Role: LEADER
I20260811 07:43:08.848261 9320 consensus_queue.cc:237] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09a6c6a726b4d6994e2826b177a5582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } }
I20260811 07:43:08.848526 9310 sys_catalog.cc:564] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:08.849843 9323 sys_catalog.cc:455] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d09a6c6a726b4d6994e2826b177a5582. Latest consensus state: current_term: 1 leader_uuid: "d09a6c6a726b4d6994e2826b177a5582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09a6c6a726b4d6994e2826b177a5582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } } }
I20260811 07:43:08.849965 9323 sys_catalog.cc:458] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:08.850354 9331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:08.850304 9322 sys_catalog.cc:455] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d09a6c6a726b4d6994e2826b177a5582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09a6c6a726b4d6994e2826b177a5582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44807 } } }
I20260811 07:43:08.850461 9322 sys_catalog.cc:458] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:08.853776 9331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:08.858747 9331 catalog_manager.cc:1347] Generated new cluster ID: 208782b2b98c4e92ad997b33c570205f
I20260811 07:43:08.858805 9331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:08.870672 9331 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:08.870966 9331 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:08.884048 9331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d09a6c6a726b4d6994e2826b177a5582: Generated new TSK 0
I20260811 07:43:08.884804 9331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:08.891427 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 9238
2026-08-11T07:43:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:08 dist-test-slave-zkpd krb5kdc[9344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:08 dist-test-slave-zkpd krb5kdc[9344](info): set up 2 sockets
Aug 11 07:43:08 dist-test-slave-zkpd krb5kdc[9344](info): commencing operation
krb5kdc: starting...
W20260811 07:43:09.965147 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:10 dist-test-slave-zkpd krb5kdc[9344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:10.090339 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:10.202399 9358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:10.202633 9358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:10.202672 9358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:10.206645 9358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:10.206694 9358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:10.206714 9358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:10.206730 9358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:10.206781 9358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:10.211732 9358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:10.212707 9358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:10.213716 9358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:10.222476 9367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:10.222433 9363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:10.222437 9364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:10.222489 9358 server_base.cc:1034] running on GCE node
I20260811 07:43:10.223403 9358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:10.224691 9358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:10.225896 9358 hybrid_clock.cc:648] HybridClock initialized: now 1786434190225874 us; error 30 us; skew 500 ppm
Aug 11 07:43:10 dist-test-slave-zkpd krb5kdc[9344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434190, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:10.228714 9358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:10.229508 9358 webserver.cc:460] Webserver started at http://127.0.0.1:41559/ using document root <none> and password file <none>
I20260811 07:43:10.230029 9358 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:10.230086 9358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:10.230245 9358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:10.232316 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "a6e35ef75f6a42c9a2e01784ff404543"
format_stamp: "Formatted at 2026-08-11 07:43:10 on dist-test-slave-zkpd"
server_key: "da4eae2f6f5fe8e7c46bb7a785432438"
server_key_iv: "57e00fa3d12e1ed1a23a0849a4406e95"
server_key_version: "encryptionkey@0"
I20260811 07:43:10.232841 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "a6e35ef75f6a42c9a2e01784ff404543"
format_stamp: "Formatted at 2026-08-11 07:43:10 on dist-test-slave-zkpd"
server_key: "da4eae2f6f5fe8e7c46bb7a785432438"
server_key_iv: "57e00fa3d12e1ed1a23a0849a4406e95"
server_key_version: "encryptionkey@0"
I20260811 07:43:10.236394 9358 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:43:10.238749 9374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:10.239985 9358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:10.240087 9358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "a6e35ef75f6a42c9a2e01784ff404543"
format_stamp: "Formatted at 2026-08-11 07:43:10 on dist-test-slave-zkpd"
server_key: "da4eae2f6f5fe8e7c46bb7a785432438"
server_key_iv: "57e00fa3d12e1ed1a23a0849a4406e95"
server_key_version: "encryptionkey@0"
I20260811 07:43:10.240166 9358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:10.252755 9358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:10.255385 9358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:10.255533 9358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:10.262422 9358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45119
I20260811 07:43:10.262511 9432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45119 every 8 connection(s)
I20260811 07:43:10.263352 9358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:10.266074 9433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:10.266567 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 9358
I20260811 07:43:10.266719 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:10.267032 18723 external_mini_cluster.cc:1428] Setting key f06484054575c2cdee419d8daf690e12
I20260811 07:43:10.273139 9433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543: Bootstrap starting.
I20260811 07:43:10.275871 9433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543: Neither blocks nor log segments found. Creating new log.
Aug 11 07:43:10 dist-test-slave-zkpd krb5kdc[9344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:43:10.276705 9433 log.cc:824] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:10.278512 9433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543: No bootstrap required, opened a new log
I20260811 07:43:10.281996 9433 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e35ef75f6a42c9a2e01784ff404543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } }
I20260811 07:43:10.282229 9433 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:10.282307 9433 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6e35ef75f6a42c9a2e01784ff404543, State: Initialized, Role: FOLLOWER
I20260811 07:43:10.282392 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:10.268464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:33129 (local address 127.0.0.1:45119)
0811 07:43:10.268945 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:10.268959 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:10.268995 (+ 36us) server_negotiation.cc:407] Connection header received
0811 07:43:10.269708 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:10.269726 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:10.270084 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:10.270374 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:10.271303 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:10.272303 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:10.273443 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:10.274014 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:10.274146 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:10.277193 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:10.277199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:10.277212 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:10.277254 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:10.279243 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:10.279813 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:10.279818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:10.279823 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:10.280012 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:10.280389 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:10.280392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:10.280394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:10.280795 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:10.281066 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:10.281358 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 07:43:10.281581 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":155,"threads_started":1}
I20260811 07:43:10.282835 9433 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e35ef75f6a42c9a2e01784ff404543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } }
I20260811 07:43:10.282989 9433 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:10.283077 9433 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:10.283176 9433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:10.284093 9433 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e35ef75f6a42c9a2e01784ff404543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } }
I20260811 07:43:10.284467 9433 leader_election.cc:302] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6e35ef75f6a42c9a2e01784ff404543; no voters:
I20260811 07:43:10.284839 9433 leader_election.cc:288] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:10.284983 9438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:10.285210 9438 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [term 1 LEADER]: Becoming Leader. State: Replica: a6e35ef75f6a42c9a2e01784ff404543, State: Running, Role: LEADER
I20260811 07:43:10.285609 9438 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e35ef75f6a42c9a2e01784ff404543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } }
I20260811 07:43:10.285698 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:10.286151 9433 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:10.288280 9439 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6e35ef75f6a42c9a2e01784ff404543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e35ef75f6a42c9a2e01784ff404543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } } }
I20260811 07:43:10.288401 9439 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:10.288730 9440 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6e35ef75f6a42c9a2e01784ff404543. Latest consensus state: current_term: 1 leader_uuid: "a6e35ef75f6a42c9a2e01784ff404543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e35ef75f6a42c9a2e01784ff404543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } } }
I20260811 07:43:10.288841 9440 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543 [sys.catalog]: This master's current role is: LEADER
W20260811 07:43:10.291846 9459 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:43:10.291965 9459 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:43:10.292084 9458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:10.292951 9458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:10.298905 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:10.289111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:39623 (local address 127.0.0.1:45119)
0811 07:43:10.289228 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:10.289232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:10.289523 (+ 291us) server_negotiation.cc:407] Connection header received
0811 07:43:10.289640 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:10.289643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:10.289691 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:10.289777 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:10.290674 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:10.291443 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:10.292595 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:10.292872 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:10.292994 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:10.294510 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:10.294514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:10.294517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:10.294542 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:10.296380 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:10.297397 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:10.297400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:10.297402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:10.297489 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:10.297896 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:10.297900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:10.297902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:10.298127 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:10.298287 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:10.298632 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 07:43:10.298741 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 07:43:10.299908 9458 catalog_manager.cc:1347] Generated new cluster ID: 1aa7965feb8e4b96a0cf309d2ad2daa5
I20260811 07:43:10.300009 9458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:10.319083 9458 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:10.320379 9458 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:10.331027 9458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6e35ef75f6a42c9a2e01784ff404543: Generated new TSK 0
I20260811 07:43:10.331755 9458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:10.342943 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 9358
W20260811 07:43:10.352689 9341 connection.cc:535] client connection to 127.0.0.1:45119 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45119): Connection reset by peer (error 104)
2026-08-11T07:43:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:10 dist-test-slave-zkpd krb5kdc[9467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:10 dist-test-slave-zkpd krb5kdc[9467](info): set up 2 sockets
Aug 11 07:43:10 dist-test-slave-zkpd krb5kdc[9467](info): commencing operation
krb5kdc: starting...
W20260811 07:43:11.440403 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:11 dist-test-slave-zkpd krb5kdc[9467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:11.566336 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:11.677681 9482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:11.677966 9482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:11.678041 9482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:11.683007 9482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:11.683063 9482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:11.683081 9482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:11.683097 9482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:11.683111 9482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:11.688288 9482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:11.689198 9482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:11.690125 9482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:11.698565 9492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:11.698576 9489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:11.698657 9488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:11.698814 9482 server_base.cc:1034] running on GCE node
I20260811 07:43:11.699431 9482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:11.700471 9482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:11.701658 9482 hybrid_clock.cc:648] HybridClock initialized: now 1786434191701627 us; error 42 us; skew 500 ppm
Aug 11 07:43:11 dist-test-slave-zkpd krb5kdc[9467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434191, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:11.708683 9482 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:11.709605 9482 webserver.cc:460] Webserver started at http://127.0.0.1:32851/ using document root <none> and password file <none>
I20260811 07:43:11.710189 9482 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:11.710282 9482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:11.710520 9482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:11.712666 9482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "1421b164ce1c44e49aab6ba254a75cef"
format_stamp: "Formatted at 2026-08-11 07:43:11 on dist-test-slave-zkpd"
server_key: "3f1546d198118266a25c89636442ce50"
server_key_iv: "9d5b8b1b473e4922329e99dfb1c49623"
server_key_version: "encryptionkey@0"
I20260811 07:43:11.713440 9482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "1421b164ce1c44e49aab6ba254a75cef"
format_stamp: "Formatted at 2026-08-11 07:43:11 on dist-test-slave-zkpd"
server_key: "3f1546d198118266a25c89636442ce50"
server_key_iv: "9d5b8b1b473e4922329e99dfb1c49623"
server_key_version: "encryptionkey@0"
I20260811 07:43:11.717228 9482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:43:11.719450 9501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:11.720839 9482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:11.721009 9482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "1421b164ce1c44e49aab6ba254a75cef"
format_stamp: "Formatted at 2026-08-11 07:43:11 on dist-test-slave-zkpd"
server_key: "3f1546d198118266a25c89636442ce50"
server_key_iv: "9d5b8b1b473e4922329e99dfb1c49623"
server_key_version: "encryptionkey@0"
I20260811 07:43:11.721122 9482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:11.745793 9482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:11.749015 9482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:11.749243 9482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:11.756464 9482 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44771
I20260811 07:43:11.756495 9553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44771 every 8 connection(s)
I20260811 07:43:11.757370 9482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:11.760114 9554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:11.762668 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 9482
I20260811 07:43:11.762821 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:11.763114 18723 external_mini_cluster.cc:1428] Setting key 153f6cfbb23ba84c8876a3494e68e47a
I20260811 07:43:11.766266 9554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef: Bootstrap starting.
I20260811 07:43:11.768590 9554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:11.769335 9554 log.cc:824] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:11.771631 9554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef: No bootstrap required, opened a new log
I20260811 07:43:11.772199 9558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:11.764580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:34649 (local address 127.0.0.1:44771)
0811 07:43:11.765014 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:11.765029 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:11.765063 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:43:11.765688 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:11.765705 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:11.766029 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:11.766381 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:11.766784 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:11.767729 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:11.768687 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:11.769145 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:11.769325 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:11.770231 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:11.770240 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:11.770260 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:11.770723 (+ 463us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:11.770787 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:11.770820 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:11.770983 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:11.771154 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 07:43:11.771281 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":131,"threads_started":1}
I20260811 07:43:11.775025 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:11.775736 9554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1421b164ce1c44e49aab6ba254a75cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44771 } }
I20260811 07:43:11.776027 9554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:11.776105 9554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1421b164ce1c44e49aab6ba254a75cef, State: Initialized, Role: FOLLOWER
I20260811 07:43:11.776568 9554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1421b164ce1c44e49aab6ba254a75cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44771 } }
I20260811 07:43:11.776697 9554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:11.776824 9554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:11.776932 9554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:11.778086 9554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1421b164ce1c44e49aab6ba254a75cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44771 } }
I20260811 07:43:11.778424 9554 leader_election.cc:302] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1421b164ce1c44e49aab6ba254a75cef; no voters:
I20260811 07:43:11.778729 9554 leader_election.cc:288] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:11.778800 9565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:11.779019 9565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [term 1 LEADER]: Becoming Leader. State: Replica: 1421b164ce1c44e49aab6ba254a75cef, State: Running, Role: LEADER
I20260811 07:43:11.779358 9565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1421b164ce1c44e49aab6ba254a75cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44771 } }
I20260811 07:43:11.780010 9554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:11.780544 9565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1421b164ce1c44e49aab6ba254a75cef. Latest consensus state: current_term: 1 leader_uuid: "1421b164ce1c44e49aab6ba254a75cef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1421b164ce1c44e49aab6ba254a75cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44771 } } }
I20260811 07:43:11.780719 9565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:11.781006 9566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1421b164ce1c44e49aab6ba254a75cef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1421b164ce1c44e49aab6ba254a75cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44771 } } }
I20260811 07:43:11.781163 9566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:11.781173 9569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:11.781632 9558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:11.777032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:53399 (local address 127.0.0.1:44771)
0811 07:43:11.777155 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:11.777159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:11.777426 (+ 267us) server_negotiation.cc:407] Connection header received
0811 07:43:11.777532 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:11.777534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:11.777570 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:11.777632 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:11.777955 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:11.778519 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:11.779597 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:11.779860 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:11.779977 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:11.781045 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:11.781048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:11.781051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:11.781063 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:11.781091 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:11.781108 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:11.781177 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:11.781475 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 07:43:11.781511 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:43:11.784713 9569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:11.790457 9569 catalog_manager.cc:1347] Generated new cluster ID: 4b9c98a95e014da89ae636eedd04c175
I20260811 07:43:11.790531 9569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:11.803177 9569 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:11.804118 9569 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:11.807924 9569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1421b164ce1c44e49aab6ba254a75cef: Generated new TSK 0
I20260811 07:43:11.808602 9569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:11.819554 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 9482
W20260811 07:43:11.827242 9463 connection.cc:535] client connection to 127.0.0.1:44771 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44771): Connection reset by peer (error 104)
2026-08-11T07:43:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1474 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:11 dist-test-slave-zkpd krb5kdc[9590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:11 dist-test-slave-zkpd krb5kdc[9590](info): set up 2 sockets
Aug 11 07:43:11 dist-test-slave-zkpd krb5kdc[9590](info): commencing operation
krb5kdc: starting...
W20260811 07:43:12.896620 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:12 dist-test-slave-zkpd krb5kdc[9590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:43:13.018527 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44615
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:13.128885 9604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:13.129158 9604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:13.129241 9604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:13.133180 9604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:13.133265 9604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:13.133306 9604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:13.133371 9604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:13.133409 9604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:13.138489 9604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44615
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:13.139546 9604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:13.140609 9604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:13.149011 9610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:13.149008 9612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:13.149008 9609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:13.149340 9604 server_base.cc:1034] running on GCE node
I20260811 07:43:13.149977 9604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:13.151033 9604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:13.152230 9604 hybrid_clock.cc:648] HybridClock initialized: now 1786434193152213 us; error 32 us; skew 500 ppm
Aug 11 07:43:13 dist-test-slave-zkpd krb5kdc[9590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:13.154878 9604 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:43:13.155705 9604 webserver.cc:460] Webserver started at http://127.0.0.1:44995/ using document root <none> and password file <none>
I20260811 07:43:13.156306 9604 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:13.156401 9604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:13.156631 9604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:13.158619 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "3ed96106990047d580463db46f32ca81"
format_stamp: "Formatted at 2026-08-11 07:43:13 on dist-test-slave-zkpd"
server_key: "b430b8b4be6539e990e4f41b3ab915cc"
server_key_iv: "6e64dd46d413a5c56804db964255ed44"
server_key_version: "encryptionkey@0"
I20260811 07:43:13.159162 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "3ed96106990047d580463db46f32ca81"
format_stamp: "Formatted at 2026-08-11 07:43:13 on dist-test-slave-zkpd"
server_key: "b430b8b4be6539e990e4f41b3ab915cc"
server_key_iv: "6e64dd46d413a5c56804db964255ed44"
server_key_version: "encryptionkey@0"
I20260811 07:43:13.162804 9604 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 07:43:13.165150 9619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:13.166337 9604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:43:13.166474 9604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "3ed96106990047d580463db46f32ca81"
format_stamp: "Formatted at 2026-08-11 07:43:13 on dist-test-slave-zkpd"
server_key: "b430b8b4be6539e990e4f41b3ab915cc"
server_key_iv: "6e64dd46d413a5c56804db964255ed44"
server_key_version: "encryptionkey@0"
I20260811 07:43:13.166579 9604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:13.171063 9604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:13.173825 9604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:13.174012 9604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:13.181613 9604 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44615
I20260811 07:43:13.181695 9676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44615 every 8 connection(s)
I20260811 07:43:13.182544 9604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:13.184458 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 9604
I20260811 07:43:13.184588 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:13.184881 18723 external_mini_cluster.cc:1428] Setting key 9e1a929e944f13c3bacede3110933fe6
I20260811 07:43:13.185972 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:13.190480 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:13.186269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:46319 (local address 127.0.0.1:44615)
0811 07:43:13.186693 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:13.186713 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:13.186754 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:43:13.187600 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:13.187624 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:13.188019 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:13.188369 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:13.189111 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:13.189117 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:13.189132 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:13.189145 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:13.189207 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:13.189234 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:13.189381 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:13.189453 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 07:43:13.189606 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":113,"threads_started":1}
I20260811 07:43:13.192921 9677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81: Bootstrap starting.
I20260811 07:43:13.193379 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:13.195713 9677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:13.196579 9677 log.cc:824] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:13.197273 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:13.195626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:46617 (local address 127.0.0.1:44615)
0811 07:43:13.195734 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:13.195739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:13.195912 (+ 173us) server_negotiation.cc:407] Connection header received
0811 07:43:13.196040 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:13.196043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:13.196076 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:13.196151 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:13.196861 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:13.196867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:13.196873 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:13.196885 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:13.196913 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:13.196930 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:13.197024 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:13.197124 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 07:43:13.197156 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:43:13.198906 9677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81: No bootstrap required, opened a new log
I20260811 07:43:13.202455 9677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed96106990047d580463db46f32ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } }
I20260811 07:43:13.202689 9677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:13.202777 9677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed96106990047d580463db46f32ca81, State: Initialized, Role: FOLLOWER
I20260811 07:43:13.203240 9677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed96106990047d580463db46f32ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } }
I20260811 07:43:13.203395 9677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:13.203480 9677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:13.203578 9677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:13.204485 9677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed96106990047d580463db46f32ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } }
I20260811 07:43:13.204818 9677 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed96106990047d580463db46f32ca81; no voters:
I20260811 07:43:13.205118 9677 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:13.205231 9688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:13.205519 9688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed96106990047d580463db46f32ca81, State: Running, Role: LEADER
I20260811 07:43:13.205883 9688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed96106990047d580463db46f32ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } }
I20260811 07:43:13.206383 9677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:13.207679 9689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ed96106990047d580463db46f32ca81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed96106990047d580463db46f32ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } } }
I20260811 07:43:13.207806 9689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:13.208123 9691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed96106990047d580463db46f32ca81. Latest consensus state: current_term: 1 leader_uuid: "3ed96106990047d580463db46f32ca81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed96106990047d580463db46f32ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } } }
I20260811 07:43:13.208264 9691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:13.208789 9699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:13.211326 9699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:13.216771 9699 catalog_manager.cc:1347] Generated new cluster ID: d7db17d4e73e48cd96e15af4ad03385c
I20260811 07:43:13.216840 9699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:13.242481 9699 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:13.242782 9699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:13.246647 9699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ed96106990047d580463db46f32ca81: Generated new TSK 0
I20260811 07:43:13.247325 9699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:13.261781 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 9604
2026-08-11T07:43:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:13 dist-test-slave-zkpd krb5kdc[9711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:13 dist-test-slave-zkpd krb5kdc[9711](info): set up 2 sockets
Aug 11 07:43:13 dist-test-slave-zkpd krb5kdc[9711](info): commencing operation
krb5kdc: starting...
W20260811 07:43:14.349040 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:14 dist-test-slave-zkpd krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:43:14.472637 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:46539
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:46539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:14.583055 9724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:14.583308 9724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:14.583348 9724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:14.587384 9724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:14.587440 9724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:14.587459 9724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:14.587476 9724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:14.587491 9724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:14.592464 9724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:46539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:46539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:14.593354 9724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:14.594312 9724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:14.602774 9730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:14.602782 9729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:14.602792 9732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:14.602970 9724 server_base.cc:1034] running on GCE node
I20260811 07:43:14.603639 9724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:14.604771 9724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:14.605937 9724 hybrid_clock.cc:648] HybridClock initialized: now 1786434194605919 us; error 32 us; skew 500 ppm
Aug 11 07:43:14 dist-test-slave-zkpd krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434194, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:14.608824 9724 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:43:14.609642 9724 webserver.cc:460] Webserver started at http://127.18.72.254:39621/ using document root <none> and password file <none>
I20260811 07:43:14.610175 9724 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:14.610231 9724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:14.610428 9724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:14.612479 9724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "c58562b9b03e4cd09edf9fc5d08bd681"
format_stamp: "Formatted at 2026-08-11 07:43:14 on dist-test-slave-zkpd"
server_key: "30ed52093cc887ef8369fc28cf00be23"
server_key_iv: "647b7ef2b3ce78ad37db7cae4ba1520b"
server_key_version: "encryptionkey@0"
I20260811 07:43:14.612978 9724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "c58562b9b03e4cd09edf9fc5d08bd681"
format_stamp: "Formatted at 2026-08-11 07:43:14 on dist-test-slave-zkpd"
server_key: "30ed52093cc887ef8369fc28cf00be23"
server_key_iv: "647b7ef2b3ce78ad37db7cae4ba1520b"
server_key_version: "encryptionkey@0"
I20260811 07:43:14.616595 9724 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:43:14.618985 9739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:14.620276 9724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:14.620388 9724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "c58562b9b03e4cd09edf9fc5d08bd681"
format_stamp: "Formatted at 2026-08-11 07:43:14 on dist-test-slave-zkpd"
server_key: "30ed52093cc887ef8369fc28cf00be23"
server_key_iv: "647b7ef2b3ce78ad37db7cae4ba1520b"
server_key_version: "encryptionkey@0"
I20260811 07:43:14.620483 9724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:14.630851 9724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:14.633916 9724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:14.634059 9724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:14.641288 9724 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:46539
I20260811 07:43:14.641310 9799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:46539 every 8 connection(s)
I20260811 07:43:14.642179 9724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:14.644994 9800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:14.649011 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 9724
I20260811 07:43:14.649150 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:14.649415 18723 external_mini_cluster.cc:1428] Setting key 1ac7782316e2adc5a943d602e52a9409
I20260811 07:43:14.650722 9800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681: Bootstrap starting.
I20260811 07:43:14.653095 9800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:14.653930 9800 log.cc:824] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:14.656056 9800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681: No bootstrap required, opened a new log
Aug 11 07:43:14 dist-test-slave-zkpd krb5kdc[9711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:43:14.660734 9800 raft_consensus.cc:348] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c58562b9b03e4cd09edf9fc5d08bd681" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46539 } }
I20260811 07:43:14.661010 9800 raft_consensus.cc:374] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:14.661072 9800 raft_consensus.cc:729] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c58562b9b03e4cd09edf9fc5d08bd681, State: Initialized, Role: FOLLOWER
I20260811 07:43:14.661580 9800 consensus_queue.cc:260] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c58562b9b03e4cd09edf9fc5d08bd681" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46539 } }
I20260811 07:43:14.661720 9800 raft_consensus.cc:388] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:14.661813 9800 raft_consensus.cc:482] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:14.661909 9800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:14.662730 9800 raft_consensus.cc:504] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c58562b9b03e4cd09edf9fc5d08bd681" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46539 } }
I20260811 07:43:14.663067 9800 leader_election.cc:302] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c58562b9b03e4cd09edf9fc5d08bd681; no voters:
I20260811 07:43:14.663393 9800 leader_election.cc:288] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:14.663460 9805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:14.663715 9805 raft_consensus.cc:686] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [term 1 LEADER]: Becoming Leader. State: Replica: c58562b9b03e4cd09edf9fc5d08bd681, State: Running, Role: LEADER
I20260811 07:43:14.664088 9805 consensus_queue.cc:237] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c58562b9b03e4cd09edf9fc5d08bd681" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46539 } }
I20260811 07:43:14.664543 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:14.650730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54913 (local address 127.18.72.254:46539)
0811 07:43:14.651154 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:14.651169 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:14.651229 (+ 60us) server_negotiation.cc:407] Connection header received
0811 07:43:14.651856 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:14.651872 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:14.652272 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:14.652539 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:14.653535 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:14.654606 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:14.655823 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:14.656455 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:14.656575 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:14.659267 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:14.659274 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:14.659298 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:14.659328 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:14.661260 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:14.661908 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:14.661914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:14.661920 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:14.662081 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:14.662458 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:14.662462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:14.662463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:14.662882 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:14.663113 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:14.663416 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 07:43:14.663653 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":134,"threads_started":1}
I20260811 07:43:14.664731 9800 sys_catalog.cc:564] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:14.665323 9805 sys_catalog.cc:455] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c58562b9b03e4cd09edf9fc5d08bd681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c58562b9b03e4cd09edf9fc5d08bd681" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46539 } } }
I20260811 07:43:14.665475 9805 sys_catalog.cc:458] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:14.665807 9806 sys_catalog.cc:455] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c58562b9b03e4cd09edf9fc5d08bd681. Latest consensus state: current_term: 1 leader_uuid: "c58562b9b03e4cd09edf9fc5d08bd681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c58562b9b03e4cd09edf9fc5d08bd681" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46539 } } }
I20260811 07:43:14.665942 9806 sys_catalog.cc:458] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:14.667080 9811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:14.667963 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:14.670383 9811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:14.676162 9811 catalog_manager.cc:1347] Generated new cluster ID: df28bc3359dc43649797bcf4f1fa5875
I20260811 07:43:14.676241 9811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:14.680755 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:14.670059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46575 (local address 127.18.72.254:46539)
0811 07:43:14.670176 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:14.670180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:14.672152 (+ 1972us) server_negotiation.cc:407] Connection header received
0811 07:43:14.672253 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:14.672256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:14.672301 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:14.672390 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:14.673186 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:14.673907 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:14.674930 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:14.675164 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:14.675277 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:14.677010 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:14.677015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:14.677021 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:14.677047 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:14.678819 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:14.679340 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:14.679351 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:14.679353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:14.679448 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:14.679786 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:14.679790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:14.679792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:14.680041 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:14.680191 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:14.680482 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 07:43:14.680585 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:43:14.693990 9811 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:14.695022 9811 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:14.701774 9811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c58562b9b03e4cd09edf9fc5d08bd681: Generated new TSK 0
I20260811 07:43:14.702512 9811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:14.711495 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 9724
W20260811 07:43:14.719746 9709 connection.cc:535] client connection to 127.18.72.254:46539 recv error: Network error: failed to read from TLS socket (remote: 127.18.72.254:46539): Connection reset by peer (error 104)
2026-08-11T07:43:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:14 dist-test-slave-zkpd krb5kdc[9833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:14 dist-test-slave-zkpd krb5kdc[9833](info): set up 2 sockets
Aug 11 07:43:14 dist-test-slave-zkpd krb5kdc[9833](info): commencing operation
krb5kdc: starting...
W20260811 07:43:15.788853 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:15 dist-test-slave-zkpd krb5kdc[9833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:43:15.910192 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:39283
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:39283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:16.021190 9847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:16.021426 9847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:16.021466 9847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:16.026104 9847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:16.026165 9847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:16.026183 9847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:16.026199 9847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:16.026255 9847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:16.031358 9847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:39283
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:39283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:16.032428 9847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:16.033444 9847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:16.041752 9857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:16.041801 9854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:16.041791 9855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:16.041859 9847 server_base.cc:1034] running on GCE node
I20260811 07:43:16.042565 9847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:16.043543 9847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:16.044698 9847 hybrid_clock.cc:648] HybridClock initialized: now 1786434196044665 us; error 43 us; skew 500 ppm
Aug 11 07:43:16 dist-test-slave-zkpd krb5kdc[9833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434196, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:16.047415 9847 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:43:16.048345 9847 webserver.cc:460] Webserver started at http://127.18.72.254:39077/ using document root <none> and password file <none>
I20260811 07:43:16.048892 9847 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:16.048954 9847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:16.049176 9847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:16.051186 9847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "e29ebc89df314d52bce03d11556a36e5"
format_stamp: "Formatted at 2026-08-11 07:43:16 on dist-test-slave-zkpd"
server_key: "2cf4167139839e567d9fd11a98a4a8d0"
server_key_iv: "87e4cd66bebac8181acbbd39ed998e64"
server_key_version: "encryptionkey@0"
I20260811 07:43:16.051748 9847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "e29ebc89df314d52bce03d11556a36e5"
format_stamp: "Formatted at 2026-08-11 07:43:16 on dist-test-slave-zkpd"
server_key: "2cf4167139839e567d9fd11a98a4a8d0"
server_key_iv: "87e4cd66bebac8181acbbd39ed998e64"
server_key_version: "encryptionkey@0"
I20260811 07:43:16.055351 9847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:43:16.057745 9865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:16.058931 9847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:43:16.059084 9847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "e29ebc89df314d52bce03d11556a36e5"
format_stamp: "Formatted at 2026-08-11 07:43:16 on dist-test-slave-zkpd"
server_key: "2cf4167139839e567d9fd11a98a4a8d0"
server_key_iv: "87e4cd66bebac8181acbbd39ed998e64"
server_key_version: "encryptionkey@0"
I20260811 07:43:16.059180 9847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:16.075186 9847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:16.077924 9847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:16.078118 9847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:16.085031 9847 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:39283
I20260811 07:43:16.085069 9922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:39283 every 8 connection(s)
I20260811 07:43:16.085908 9847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:16.086325 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 9847
I20260811 07:43:16.086505 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:16.086807 18723 external_mini_cluster.cc:1428] Setting key 06de3c5b13a9b47c57b5fb30b28e82fa
I20260811 07:43:16.088855 9923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:16.094084 9923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5: Bootstrap starting.
I20260811 07:43:16.095297 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:16.088179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40601 (local address 127.18.72.254:39283)
0811 07:43:16.088574 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:16.088598 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:16.088632 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:43:16.089294 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:16.089310 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:16.089618 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:16.089941 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:16.090309 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:16.091230 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:16.092183 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:16.092639 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:16.092814 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:16.093666 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:16.093670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:16.093682 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:16.094056 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:16.094104 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:16.094126 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:16.094269 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:16.094491 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 07:43:16.094631 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":140,"threads_started":1}
I20260811 07:43:16.096398 9923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:16.097131 9923 log.cc:824] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:16.098096 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:16.099051 9923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5: No bootstrap required, opened a new log
I20260811 07:43:16.102634 9923 raft_consensus.cc:348] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29ebc89df314d52bce03d11556a36e5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39283 } }
I20260811 07:43:16.102890 9923 raft_consensus.cc:374] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:16.102985 9923 raft_consensus.cc:729] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e29ebc89df314d52bce03d11556a36e5, State: Initialized, Role: FOLLOWER
I20260811 07:43:16.103513 9923 consensus_queue.cc:260] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29ebc89df314d52bce03d11556a36e5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39283 } }
I20260811 07:43:16.103722 9923 raft_consensus.cc:388] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:16.103794 9923 raft_consensus.cc:482] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:16.103907 9923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:16.104786 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:16.100134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41277 (local address 127.18.72.254:39283)
0811 07:43:16.100251 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:16.100256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:16.100488 (+ 232us) server_negotiation.cc:407] Connection header received
0811 07:43:16.100595 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:16.100597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:16.100631 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:16.100694 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:16.100985 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:16.101903 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:16.103019 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:16.103290 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:16.103370 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:16.104303 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:16.104306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:16.104308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:16.104318 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:16.104341 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:16.104355 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:16.104423 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:16.104601 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 07:43:16.104650 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:43:16.105098 9923 raft_consensus.cc:504] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29ebc89df314d52bce03d11556a36e5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39283 } }
I20260811 07:43:16.105444 9923 leader_election.cc:302] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e29ebc89df314d52bce03d11556a36e5; no voters:
I20260811 07:43:16.105794 9923 leader_election.cc:288] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:16.105896 9934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:16.106123 9934 raft_consensus.cc:686] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [term 1 LEADER]: Becoming Leader. State: Replica: e29ebc89df314d52bce03d11556a36e5, State: Running, Role: LEADER
I20260811 07:43:16.106417 9934 consensus_queue.cc:237] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29ebc89df314d52bce03d11556a36e5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39283 } }
I20260811 07:43:16.107028 9923 sys_catalog.cc:564] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:16.108026 9935 sys_catalog.cc:455] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e29ebc89df314d52bce03d11556a36e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29ebc89df314d52bce03d11556a36e5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39283 } } }
I20260811 07:43:16.108055 9936 sys_catalog.cc:455] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e29ebc89df314d52bce03d11556a36e5. Latest consensus state: current_term: 1 leader_uuid: "e29ebc89df314d52bce03d11556a36e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29ebc89df314d52bce03d11556a36e5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 39283 } } }
I20260811 07:43:16.108145 9935 sys_catalog.cc:458] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:16.108161 9936 sys_catalog.cc:458] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:16.109244 9943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:16.112082 9943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:16.117314 9943 catalog_manager.cc:1347] Generated new cluster ID: d414c90004ac4ddf896e44222b55aa54
I20260811 07:43:16.117398 9943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:16.127959 9943 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:16.128808 9943 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:16.138442 9943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e29ebc89df314d52bce03d11556a36e5: Generated new TSK 0
I20260811 07:43:16.139127 9943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:16.143821 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 9847
W20260811 07:43:16.152257 9831 connection.cc:535] client connection to 127.18.72.254:39283 recv error: Network error: failed to read from TLS socket (remote: 127.18.72.254:39283): Connection reset by peer (error 104)
2026-08-11T07:43:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:16 dist-test-slave-zkpd krb5kdc[9955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:16 dist-test-slave-zkpd krb5kdc[9955](info): set up 2 sockets
Aug 11 07:43:16 dist-test-slave-zkpd krb5kdc[9955](info): commencing operation
krb5kdc: starting...
W20260811 07:43:17.220434 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:17 dist-test-slave-zkpd krb5kdc[9955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:43:17.349073 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:46421
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:46421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:17.465449 9970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:17.465807 9970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:17.465910 9970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:17.471707 9970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:17.471791 9970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:17.471828 9970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:17.471865 9970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:17.471906 9970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:17.477447 9970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:46421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:46421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:17.478565 9970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:17.479636 9970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:17.488310 9975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:17.488322 9978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:17.488318 9976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:17.488546 9970 server_base.cc:1034] running on GCE node
I20260811 07:43:17.489187 9970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:17.490233 9970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:17.491429 9970 hybrid_clock.cc:648] HybridClock initialized: now 1786434197491410 us; error 30 us; skew 500 ppm
Aug 11 07:43:17 dist-test-slave-zkpd krb5kdc[9955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434197, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:17.498431 9970 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:43:17.499279 9970 webserver.cc:460] Webserver started at http://127.18.72.254:44323/ using document root <none> and password file <none>
I20260811 07:43:17.499846 9970 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:17.499966 9970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:17.500211 9970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:17.502374 9970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "45efa23c7347471da6c75a205a9488ef"
format_stamp: "Formatted at 2026-08-11 07:43:17 on dist-test-slave-zkpd"
server_key: "47bdd25469b773ba6b2763735c2e371a"
server_key_iv: "3e0fd0fe24d15b7af4231cfd1c3f6551"
server_key_version: "encryptionkey@0"
I20260811 07:43:17.502945 9970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "45efa23c7347471da6c75a205a9488ef"
format_stamp: "Formatted at 2026-08-11 07:43:17 on dist-test-slave-zkpd"
server_key: "47bdd25469b773ba6b2763735c2e371a"
server_key_iv: "3e0fd0fe24d15b7af4231cfd1c3f6551"
server_key_version: "encryptionkey@0"
I20260811 07:43:17.506675 9970 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260811 07:43:17.509079 9985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:17.510246 9970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:43:17.510392 9970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "45efa23c7347471da6c75a205a9488ef"
format_stamp: "Formatted at 2026-08-11 07:43:17 on dist-test-slave-zkpd"
server_key: "47bdd25469b773ba6b2763735c2e371a"
server_key_iv: "3e0fd0fe24d15b7af4231cfd1c3f6551"
server_key_version: "encryptionkey@0"
I20260811 07:43:17.510504 9970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:17.514998 9970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:17.517874 9970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:17.518059 9970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:17.525343 9970 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:46421
I20260811 07:43:17.525341 10041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:46421 every 8 connection(s)
I20260811 07:43:17.526293 9970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:17.529073 10042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:17.534539 10042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef: Bootstrap starting.
I20260811 07:43:17.535759 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 9970
I20260811 07:43:17.536031 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:17.536317 18723 external_mini_cluster.cc:1428] Setting key 6d97f87e439d5990410d495976041d30
I20260811 07:43:17.537254 10042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:17.537987 10042 log.cc:824] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:17.540117 10042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef: No bootstrap required, opened a new log
I20260811 07:43:17.541512 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:17.537410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34057 (local address 127.18.72.254:46421)
0811 07:43:17.537885 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:17.537901 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:17.537935 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:43:17.538780 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:17.538800 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:17.539063 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:17.539337 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:17.540261 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:17.540266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:17.540278 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:17.540294 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:17.540347 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:17.540367 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:17.540472 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:17.540633 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 07:43:17.540804 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":156,"threads_started":1}
I20260811 07:43:17.543951 10042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45efa23c7347471da6c75a205a9488ef" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46421 } }
I20260811 07:43:17.544209 10042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:17.544240 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:17.544291 10042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45efa23c7347471da6c75a205a9488ef, State: Initialized, Role: FOLLOWER
I20260811 07:43:17.544683 10042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45efa23c7347471da6c75a205a9488ef" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46421 } }
I20260811 07:43:17.544808 10042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:17.544890 10042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:17.544998 10042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:17.545994 10042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45efa23c7347471da6c75a205a9488ef" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46421 } }
I20260811 07:43:17.546365 10042 leader_election.cc:302] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45efa23c7347471da6c75a205a9488ef; no voters:
I20260811 07:43:17.546715 10042 leader_election.cc:288] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:17.546782 10052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:17.547019 10052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [term 1 LEADER]: Becoming Leader. State: Replica: 45efa23c7347471da6c75a205a9488ef, State: Running, Role: LEADER
I20260811 07:43:17.547358 10052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45efa23c7347471da6c75a205a9488ef" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46421 } }
I20260811 07:43:17.548074 10042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:17.548467 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:17.546397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42771 (local address 127.18.72.254:46421)
0811 07:43:17.546515 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:17.546519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:17.546780 (+ 261us) server_negotiation.cc:407] Connection header received
0811 07:43:17.546954 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:17.546958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:17.546999 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:17.547085 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:17.547965 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:17.547969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:17.547972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:17.547984 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:17.548011 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:17.548024 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:17.548116 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:17.548301 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 07:43:17.548339 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:43:17.548738 10052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45efa23c7347471da6c75a205a9488ef. Latest consensus state: current_term: 1 leader_uuid: "45efa23c7347471da6c75a205a9488ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45efa23c7347471da6c75a205a9488ef" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46421 } } }
I20260811 07:43:17.548853 10052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:17.549826 10053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45efa23c7347471da6c75a205a9488ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45efa23c7347471da6c75a205a9488ef" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 46421 } } }
I20260811 07:43:17.549966 10053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef [sys.catalog]: This master's current role is: LEADER
W20260811 07:43:17.552798 10068 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:43:17.552860 10068 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:43:17.552933 10065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:17.553601 10065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:17.558990 10065 catalog_manager.cc:1347] Generated new cluster ID: 0171d00a7f184150a7cfbddcdee891f2
I20260811 07:43:17.559051 10065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:17.579383 10065 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:17.579764 10065 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:17.592248 10065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45efa23c7347471da6c75a205a9488ef: Generated new TSK 0
I20260811 07:43:17.593284 10065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:17.604790 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 9970
2026-08-11T07:43:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:17 dist-test-slave-zkpd krb5kdc[10076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:17 dist-test-slave-zkpd krb5kdc[10076](info): set up 2 sockets
Aug 11 07:43:17 dist-test-slave-zkpd krb5kdc[10076](info): commencing operation
krb5kdc: starting...
W20260811 07:43:18.697949 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:18 dist-test-slave-zkpd krb5kdc[10076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:43:18.834620 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:43187
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:34603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:43187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:18.951643 10089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:18.951996 10089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:18.952086 10089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:18.956105 10089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:18.956214 10089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:18.956251 10089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:18.956290 10089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:18.956326 10089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:18.961687 10089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:34603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:43187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:18.962839 10089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:18.964017 10089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:18.973215 10094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:18.973215 10097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:18.973237 10095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:18.973452 10089 server_base.cc:1034] running on GCE node
I20260811 07:43:18.974280 10089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:18.975525 10089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:18.976796 10089 hybrid_clock.cc:648] HybridClock initialized: now 1786434198976739 us; error 75 us; skew 500 ppm
Aug 11 07:43:18 dist-test-slave-zkpd krb5kdc[10076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434198, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:18.979733 10089 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:43:18.980746 10089 webserver.cc:460] Webserver started at http://127.18.72.254:46563/ using document root <none> and password file <none>
I20260811 07:43:18.981390 10089 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:18.981494 10089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:18.981745 10089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:18.983978 10089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "e22feb4773c6477fa02f84fdfd5589b3"
format_stamp: "Formatted at 2026-08-11 07:43:18 on dist-test-slave-zkpd"
server_key: "c7a5a8172de631fbd39df53044288617"
server_key_iv: "6be6ecfca46b9b7009e0d6923d4a5b66"
server_key_version: "encryptionkey@0"
I20260811 07:43:18.984617 10089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "e22feb4773c6477fa02f84fdfd5589b3"
format_stamp: "Formatted at 2026-08-11 07:43:18 on dist-test-slave-zkpd"
server_key: "c7a5a8172de631fbd39df53044288617"
server_key_iv: "6be6ecfca46b9b7009e0d6923d4a5b66"
server_key_version: "encryptionkey@0"
I20260811 07:43:18.988660 10089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 07:43:18.991325 10105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:18.992693 10089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:43:18.992846 10089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "e22feb4773c6477fa02f84fdfd5589b3"
format_stamp: "Formatted at 2026-08-11 07:43:18 on dist-test-slave-zkpd"
server_key: "c7a5a8172de631fbd39df53044288617"
server_key_iv: "6be6ecfca46b9b7009e0d6923d4a5b66"
server_key_version: "encryptionkey@0"
I20260811 07:43:18.992967 10089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:19.005031 10089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:19.008278 10089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:19.008491 10089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:19.015847 10089 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:43187
I20260811 07:43:19.015954 10160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:43187 every 8 connection(s)
I20260811 07:43:19.016877 10089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:19.019761 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:19.022272 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 10089
I20260811 07:43:19.022387 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:19.022670 18723 external_mini_cluster.cc:1428] Setting key ed8f823d07cc1bd1f9b7df1a6e02ac3d
I20260811 07:43:19.025990 10161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3: Bootstrap starting.
I20260811 07:43:19.028714 10161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:19.029748 10161 log.cc:824] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:19.031775 10161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3: No bootstrap required, opened a new log
Aug 11 07:43:19 dist-test-slave-zkpd krb5kdc[10076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:43:19.036283 10161 raft_consensus.cc:348] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22feb4773c6477fa02f84fdfd5589b3" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43187 } }
I20260811 07:43:19.036608 10161 raft_consensus.cc:374] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:19.036664 10161 raft_consensus.cc:729] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e22feb4773c6477fa02f84fdfd5589b3, State: Initialized, Role: FOLLOWER
I20260811 07:43:19.037204 10161 consensus_queue.cc:260] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22feb4773c6477fa02f84fdfd5589b3" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43187 } }
I20260811 07:43:19.037386 10161 raft_consensus.cc:388] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:19.037498 10161 raft_consensus.cc:482] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:19.037614 10161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:19.038841 10161 raft_consensus.cc:504] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22feb4773c6477fa02f84fdfd5589b3" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43187 } }
I20260811 07:43:19.038929 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:19.024192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55969 (local address 127.18.72.254:43187)
0811 07:43:19.024608 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:19.024623 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:19.024660 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:43:19.025292 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:19.025308 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:19.025653 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:19.025921 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:19.026933 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:19.028072 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:19.029425 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:19.029947 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:19.030071 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:19.032795 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:19.032802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:19.032820 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:19.032852 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:19.035494 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:19.036038 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:19.036045 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:19.036051 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:19.036196 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:19.036488 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:19.036492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:19.036494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:19.037052 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:19.037263 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:19.037566 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 07:43:19.037914 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
I20260811 07:43:19.039233 10161 leader_election.cc:302] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e22feb4773c6477fa02f84fdfd5589b3; no voters:
I20260811 07:43:19.039584 10161 leader_election.cc:288] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:19.039714 10167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:19.040026 10167 raft_consensus.cc:686] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [term 1 LEADER]: Becoming Leader. State: Replica: e22feb4773c6477fa02f84fdfd5589b3, State: Running, Role: LEADER
I20260811 07:43:19.040343 10167 consensus_queue.cc:237] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22feb4773c6477fa02f84fdfd5589b3" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43187 } }
I20260811 07:43:19.040926 10161 sys_catalog.cc:564] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:19.041718 10168 sys_catalog.cc:455] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e22feb4773c6477fa02f84fdfd5589b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22feb4773c6477fa02f84fdfd5589b3" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43187 } } }
I20260811 07:43:19.041846 10168 sys_catalog.cc:458] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:19.042544 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:19.042936 10169 sys_catalog.cc:455] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e22feb4773c6477fa02f84fdfd5589b3. Latest consensus state: current_term: 1 leader_uuid: "e22feb4773c6477fa02f84fdfd5589b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22feb4773c6477fa02f84fdfd5589b3" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 43187 } } }
I20260811 07:43:19.043030 10169 sys_catalog.cc:458] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:19.043467 10175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:19.046705 10175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:19.052553 10175 catalog_manager.cc:1347] Generated new cluster ID: da84fe6c2819447bac73a1213b0ba54f
I20260811 07:43:19.052609 10175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:19.053648 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:19.044854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51967 (local address 127.18.72.254:43187)
0811 07:43:19.044953 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:19.044957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:19.045212 (+ 255us) server_negotiation.cc:407] Connection header received
0811 07:43:19.045328 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:19.045331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:19.045376 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:19.045458 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:19.046270 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:19.047035 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:19.048226 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:19.048548 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:19.048662 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:19.050070 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:19.050074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:19.050077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:19.050100 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:19.051752 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:19.052267 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:19.052270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:19.052272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:19.052361 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:19.052703 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:19.052706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:19.052708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:19.052948 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:19.053082 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:19.053357 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 07:43:19.053480 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 07:43:19.067235 10175 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:19.068234 10175 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:19.073747 10175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e22feb4773c6477fa02f84fdfd5589b3: Generated new TSK 0
I20260811 07:43:19.074426 10175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:19.079255 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 10089
2026-08-11T07:43:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1474 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:19 dist-test-slave-zkpd krb5kdc[10196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:19 dist-test-slave-zkpd krb5kdc[10196](info): set up 2 sockets
Aug 11 07:43:19 dist-test-slave-zkpd krb5kdc[10196](info): commencing operation
krb5kdc: starting...
W20260811 07:43:20.161206 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:20 dist-test-slave-zkpd krb5kdc[10196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:43:20.291219 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45729
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:45729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:20.405288 10211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:20.405594 10211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:20.405670 10211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:20.409775 10211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:20.409862 10211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:20.409900 10211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:20.409935 10211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:20.409972 10211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:20.415302 10211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:45729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:45729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:20.416471 10211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:20.417517 10211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:20.425988 10220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:20.426007 10218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:20.425976 10217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:20.426437 10211 server_base.cc:1034] running on GCE node
I20260811 07:43:20.427048 10211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:20.428225 10211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:20.429415 10211 hybrid_clock.cc:648] HybridClock initialized: now 1786434200429403 us; error 25 us; skew 500 ppm
Aug 11 07:43:20 dist-test-slave-zkpd krb5kdc[10196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434200, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:20.432188 10211 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:43:20.433111 10211 webserver.cc:460] Webserver started at http://127.18.72.254:32997/ using document root <none> and password file <none>
I20260811 07:43:20.433682 10211 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:20.433789 10211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:20.434058 10211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:20.436229 10211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "144f76c490964e30b5c7e9b467b66623"
format_stamp: "Formatted at 2026-08-11 07:43:20 on dist-test-slave-zkpd"
server_key: "f339ef03dad523b7e758a2ca64a4b33e"
server_key_iv: "fe65cdd7c302a811706bfa4f7c693e34"
server_key_version: "encryptionkey@0"
I20260811 07:43:20.436817 10211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "144f76c490964e30b5c7e9b467b66623"
format_stamp: "Formatted at 2026-08-11 07:43:20 on dist-test-slave-zkpd"
server_key: "f339ef03dad523b7e758a2ca64a4b33e"
server_key_iv: "fe65cdd7c302a811706bfa4f7c693e34"
server_key_version: "encryptionkey@0"
I20260811 07:43:20.440642 10211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 07:43:20.443025 10227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:20.444293 10211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:20.444435 10211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "144f76c490964e30b5c7e9b467b66623"
format_stamp: "Formatted at 2026-08-11 07:43:20 on dist-test-slave-zkpd"
server_key: "f339ef03dad523b7e758a2ca64a4b33e"
server_key_iv: "fe65cdd7c302a811706bfa4f7c693e34"
server_key_version: "encryptionkey@0"
I20260811 07:43:20.444547 10211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:20.458931 10211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:20.461688 10211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:20.461900 10211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:20.469863 10211 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:45729
I20260811 07:43:20.469880 10288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:45729 every 8 connection(s)
I20260811 07:43:20.470804 10211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:20.473668 10290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:20.477667 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 10211
I20260811 07:43:20.477813 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:20.478137 18723 external_mini_cluster.cc:1428] Setting key d913c529f0ff099dcd7288e04e8e9914
I20260811 07:43:20.480137 10290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623: Bootstrap starting.
I20260811 07:43:20.483013 10290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:20.483942 10290 log.cc:824] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:20.486274 10290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623: No bootstrap required, opened a new log
I20260811 07:43:20.489061 10293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:20.479595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40955 (local address 127.18.72.254:45729)
0811 07:43:20.480055 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:20.480077 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:20.480121 (+ 44us) server_negotiation.cc:407] Connection header received
0811 07:43:20.480969 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:20.480994 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:20.481456 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:20.481834 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:20.482254 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:20.483768 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:20.485079 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:20.485756 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:20.485930 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:20.486802 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:20.486808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:20.486825 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:20.487308 (+ 483us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:20.487382 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:20.487413 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:20.487522 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:20.487628 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 07:43:20.487836 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":106,"threads_started":1}
I20260811 07:43:20.490311 10290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f76c490964e30b5c7e9b467b66623" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45729 } }
I20260811 07:43:20.490552 10290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:20.490643 10290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144f76c490964e30b5c7e9b467b66623, State: Initialized, Role: FOLLOWER
I20260811 07:43:20.491155 10290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f76c490964e30b5c7e9b467b66623" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45729 } }
I20260811 07:43:20.491333 10290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:20.491400 10290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:20.491503 10290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:20.492008 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:20.492545 10290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f76c490964e30b5c7e9b467b66623" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45729 } }
I20260811 07:43:20.492897 10290 leader_election.cc:302] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 144f76c490964e30b5c7e9b467b66623; no voters:
I20260811 07:43:20.493289 10290 leader_election.cc:288] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:20.493346 10297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:20.493567 10297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [term 1 LEADER]: Becoming Leader. State: Replica: 144f76c490964e30b5c7e9b467b66623, State: Running, Role: LEADER
I20260811 07:43:20.493942 10297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f76c490964e30b5c7e9b467b66623" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45729 } }
I20260811 07:43:20.494613 10290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:20.495296 10297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 144f76c490964e30b5c7e9b467b66623. Latest consensus state: current_term: 1 leader_uuid: "144f76c490964e30b5c7e9b467b66623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f76c490964e30b5c7e9b467b66623" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45729 } } }
I20260811 07:43:20.496109 10297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:20.495999 10300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "144f76c490964e30b5c7e9b467b66623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144f76c490964e30b5c7e9b467b66623" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 45729 } } }
I20260811 07:43:20.496445 10300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:20.497184 10309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:20.498672 10293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:20.494106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34513 (local address 127.18.72.254:45729)
0811 07:43:20.494204 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:20.494209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:20.494410 (+ 201us) server_negotiation.cc:407] Connection header received
0811 07:43:20.494527 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:20.494530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:20.494589 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:20.494651 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:20.494996 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:20.495731 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:20.496883 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:20.497219 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:20.497301 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:20.498272 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:20.498276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:20.498278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:20.498291 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:20.498319 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:20.498332 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:20.498409 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:20.498495 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 07:43:20.498549 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":28}
I20260811 07:43:20.500501 10309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:20.505720 10309 catalog_manager.cc:1347] Generated new cluster ID: a227ef13fabd4dd4926b686a1518c8c8
I20260811 07:43:20.505781 10309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:20.530843 10309 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:20.531853 10309 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:20.536191 10309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 144f76c490964e30b5c7e9b467b66623: Generated new TSK 0
I20260811 07:43:20.536846 10309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:20.543334 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 10211
2026-08-11T07:43:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1464 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:20 dist-test-slave-zkpd krb5kdc[10322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:20 dist-test-slave-zkpd krb5kdc[10322](info): set up 2 sockets
Aug 11 07:43:20 dist-test-slave-zkpd krb5kdc[10322](info): commencing operation
krb5kdc: starting...
W20260811 07:43:21.613312 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:21 dist-test-slave-zkpd krb5kdc[10322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:43:21.745051 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:35789
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:32983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:35789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:21.858589 10336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:21.858855 10336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:21.858894 10336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:21.862861 10336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:21.862922 10336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:21.862941 10336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:21.862959 10336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:21.862974 10336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:21.868368 10336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:32983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:35789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:35789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:21.869447 10336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:21.870487 10336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 07:43:21.879232 10336 server_base.cc:1034] running on GCE node
W20260811 07:43:21.879217 10345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:21.879217 10342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:21.879204 10343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:21.880172 10336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:21.881198 10336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:21.882365 10336 hybrid_clock.cc:648] HybridClock initialized: now 1786434201882345 us; error 31 us; skew 500 ppm
Aug 11 07:43:21 dist-test-slave-zkpd krb5kdc[10322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434201, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:21.885103 10336 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:43:21.886006 10336 webserver.cc:460] Webserver started at http://127.18.72.254:40725/ using document root <none> and password file <none>
I20260811 07:43:21.886569 10336 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:21.886628 10336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:21.886838 10336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:21.888976 10336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "23575bafc2254436ac3ce4e2399e9bb5"
format_stamp: "Formatted at 2026-08-11 07:43:21 on dist-test-slave-zkpd"
server_key: "6c1ac043ca67f517d32a7027ce0d59ee"
server_key_iv: "ef4899ccedd5a6fab00c766359ec4ab0"
server_key_version: "encryptionkey@0"
I20260811 07:43:21.889562 10336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "23575bafc2254436ac3ce4e2399e9bb5"
format_stamp: "Formatted at 2026-08-11 07:43:21 on dist-test-slave-zkpd"
server_key: "6c1ac043ca67f517d32a7027ce0d59ee"
server_key_iv: "ef4899ccedd5a6fab00c766359ec4ab0"
server_key_version: "encryptionkey@0"
I20260811 07:43:21.893252 10336 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:43:21.895653 10354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:21.896983 10336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:21.897127 10336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "23575bafc2254436ac3ce4e2399e9bb5"
format_stamp: "Formatted at 2026-08-11 07:43:21 on dist-test-slave-zkpd"
server_key: "6c1ac043ca67f517d32a7027ce0d59ee"
server_key_iv: "ef4899ccedd5a6fab00c766359ec4ab0"
server_key_version: "encryptionkey@0"
I20260811 07:43:21.897239 10336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:21.901867 10336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:21.904808 10336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:21.904999 10336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:21.913007 10336 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:35789
I20260811 07:43:21.913038 10411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:35789 every 8 connection(s)
I20260811 07:43:21.914003 10336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:21.916926 10412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:21.921772 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 10336
I20260811 07:43:21.921937 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:21.922240 18723 external_mini_cluster.cc:1428] Setting key 4630ea69e04ddf3df9005a0de42773c4
I20260811 07:43:21.922885 10412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5: Bootstrap starting.
I20260811 07:43:21.925295 10412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:21.926177 10412 log.cc:824] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:21.927356 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:21.923607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37165 (local address 127.18.72.254:35789)
0811 07:43:21.924095 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:21.924110 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:21.924144 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:43:21.924759 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:21.924776 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:21.925063 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:21.925393 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:21.926198 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:21.926203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:21.926215 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:21.926225 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:21.926277 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:21.926318 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:21.926421 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:21.926532 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 07:43:21.926692 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":140,"threads_started":1}
I20260811 07:43:21.928294 10412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5: No bootstrap required, opened a new log
I20260811 07:43:21.930115 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:21.932183 10412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23575bafc2254436ac3ce4e2399e9bb5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35789 } }
I20260811 07:43:21.932475 10412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:43:21.932543 10412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23575bafc2254436ac3ce4e2399e9bb5, State: Initialized, Role: FOLLOWER
I20260811 07:43:21.932999 10412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23575bafc2254436ac3ce4e2399e9bb5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35789 } }
I20260811 07:43:21.933194 10412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:43:21.933269 10412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:43:21.933394 10412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:21.933944 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:21.932164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56227 (local address 127.18.72.254:35789)
0811 07:43:21.932291 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:21.932295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:21.932480 (+ 185us) server_negotiation.cc:407] Connection header received
0811 07:43:21.932678 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:21.932681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:21.932718 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:21.932810 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:21.933549 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:21.933554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:21.933560 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:43:21.933573 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:21.933602 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:43:21.933619 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:21.933728 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:21.933795 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 07:43:21.933830 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:43:21.934475 10412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23575bafc2254436ac3ce4e2399e9bb5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35789 } }
I20260811 07:43:21.934821 10412 leader_election.cc:302] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23575bafc2254436ac3ce4e2399e9bb5; no voters:
I20260811 07:43:21.935150 10412 leader_election.cc:288] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:43:21.935235 10424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:21.935470 10424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [term 1 LEADER]: Becoming Leader. State: Replica: 23575bafc2254436ac3ce4e2399e9bb5, State: Running, Role: LEADER
I20260811 07:43:21.935760 10424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23575bafc2254436ac3ce4e2399e9bb5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35789 } }
I20260811 07:43:21.936401 10412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:21.937562 10426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23575bafc2254436ac3ce4e2399e9bb5. Latest consensus state: current_term: 1 leader_uuid: "23575bafc2254436ac3ce4e2399e9bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23575bafc2254436ac3ce4e2399e9bb5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35789 } } }
I20260811 07:43:21.937578 10425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23575bafc2254436ac3ce4e2399e9bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23575bafc2254436ac3ce4e2399e9bb5" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 35789 } } }
I20260811 07:43:21.937718 10426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:21.937723 10425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:21.938146 10435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:21.941294 10435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:21.947176 10435 catalog_manager.cc:1347] Generated new cluster ID: fd4ba423efbd4f9ab5a30da7a0949937
I20260811 07:43:21.947248 10435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:21.966519 10435 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:21.966902 10435 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:21.973126 10435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23575bafc2254436ac3ce4e2399e9bb5: Generated new TSK 0
I20260811 07:43:21.973816 10435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:43:21.978873 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 10336
2026-08-11T07:43:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1435 ms)
[----------] 18 tests from AuthTokenIssuingTest (26267 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:22 dist-test-slave-zkpd krb5kdc[10449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:22 dist-test-slave-zkpd krb5kdc[10449](info): set up 2 sockets
Aug 11 07:43:22 dist-test-slave-zkpd krb5kdc[10449](info): commencing operation
krb5kdc: starting...
W20260811 07:43:23.054239 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:43:23.179067 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:40237
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:33129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:40237,127.18.72.253:32951,127.18.72.252:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:23.291160 10463 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:43:23.291435 10463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:23.291465 10463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:23.291503 10463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:23.295503 10463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:23.295552 10463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:23.295571 10463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:23.295588 10463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:23.295603 10463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:23.300752 10463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:33129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:40237,127.18.72.253:32951,127.18.72.252:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:40237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:23.301783 10463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:23.302814 10463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:23.311323 10471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:23.311334 10469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:23.311364 10468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:23.311514 10463 server_base.cc:1034] running on GCE node
I20260811 07:43:23.312234 10463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:23.313300 10463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:23.314466 10463 hybrid_clock.cc:648] HybridClock initialized: now 1786434203314446 us; error 29 us; skew 500 ppm
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:23.320473 10463 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:43:23.321398 10463 webserver.cc:460] Webserver started at http://127.18.72.254:36121/ using document root <none> and password file <none>
I20260811 07:43:23.322005 10463 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:23.322070 10463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:23.322329 10463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:23.324468 10463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "b2d96a794eac45f1a108e1a190843382"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "5e7e7ad22b5287e5c6cf89d652cd9243"
server_key_iv: "369c695ff3293560ccbbd4e4ff867f5b"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.325031 10463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "b2d96a794eac45f1a108e1a190843382"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "5e7e7ad22b5287e5c6cf89d652cd9243"
server_key_iv: "369c695ff3293560ccbbd4e4ff867f5b"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.328751 10463 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:43:23.331087 10479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:23.332567 10463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:23.332716 10463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "b2d96a794eac45f1a108e1a190843382"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "5e7e7ad22b5287e5c6cf89d652cd9243"
server_key_iv: "369c695ff3293560ccbbd4e4ff867f5b"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.332847 10463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:23.363402 10463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:23.367306 10463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:23.367520 10463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:23.374665 10463 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:40237
I20260811 07:43:23.374974 10538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:40237 every 8 connection(s)
I20260811 07:43:23.375682 10463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:23.378526 10539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:23.382246 10539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:23.385510 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 10463
I20260811 07:43:23.385651 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:23.385952 18723 external_mini_cluster.cc:1428] Setting key 745450f80178adcfece5a3fc78e7b869
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.253@KRBTEST.COM" created.
I20260811 07:43:23.398423 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.384493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:40237 (local address 127.0.0.1:46664)
0811 07:43:23.385006 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.385058 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.386173 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.386933 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.386942 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.387438 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.387832 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.387851 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.389593 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.389596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.390299 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.390308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.391312 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.391315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.391446 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.392107 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.392123 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.393524 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.395773 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.395781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.395789 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.396072 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.396392 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.396395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.396396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.396501 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.397084 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.397089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.397208 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.397335 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.397593 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":4,"thread_start_us":123,"threads_started":1}
I20260811 07:43:23.398428 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.384493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.18.72.254:40237)
0811 07:43:23.384969 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.384991 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.385159 (+ 168us) server_negotiation.cc:407] Connection header received
0811 07:43:23.386522 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.386528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.386710 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.386883 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.388035 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.389468 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.390439 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.391130 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.391306 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.393675 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.393683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.393711 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.393742 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.395640 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.396189 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.396194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.396195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.396305 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.396612 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.396615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.396616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.396980 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.397082 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.397427 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.397670 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":366,"spinlock_wait_cycles":26496,"thread_start_us":204,"threads_started":1}
I20260811 07:43:23.400974 10539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:23.401647 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.401422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43336)
0811 07:43:23.401533 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.401597 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 07:43:23.401923 10483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
I20260811 07:43:23.402877 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.402597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43352)
0811 07:43:23.402726 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.402816 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 07:43:23.403102 10539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:32951: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab.
Entry for principal kudu/127.18.72.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab.
I20260811 07:43:23.414300 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.253:32951
--webserver_interface=127.18.72.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab
--principal=kudu/127.18.72.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:33129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:40237,127.18.72.253:32951,127.18.72.252:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:43:23.452343 10539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } attempt: 1
I20260811 07:43:23.453251 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.452953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43362)
0811 07:43:23.453112 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.453175 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 07:43:23.454003 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.453775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43364)
0811 07:43:23.453887 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.453944 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 07:43:23.454139 10539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:32951: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
I20260811 07:43:23.522294 10539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } attempt: 2
I20260811 07:43:23.523100 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.522811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43376)
0811 07:43:23.522959 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.523028 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 07:43:23.523813 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.523597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43392)
0811 07:43:23.523704 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.523751 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 07:43:23.523964 10539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:32951: Network error: Client connection negotiation failed: client connection to 127.18.72.253:32951: connect: Connection refused (error 111)
W20260811 07:43:23.532070 10547 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:43:23.532364 10547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:23.532425 10547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:23.532500 10547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:23.536491 10547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:23.536562 10547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:23.536607 10547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:23.536662 10547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:23.536697 10547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:23.541913 10547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:33129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:40237,127.18.72.253:32951,127.18.72.252:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab
--principal=kudu/127.18.72.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.253:32951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.72.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:23.542966 10547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:23.544055 10547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:23.552217 10553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:23.552235 10552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:23.552425 10547 server_base.cc:1034] running on GCE node
W20260811 07:43:23.552526 10556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:23.553030 10547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:23.554068 10547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:23.555229 10547 hybrid_clock.cc:648] HybridClock initialized: now 1786434203555210 us; error 28 us; skew 500 ppm
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:23.557971 10547 init.cc:377] Logged in from keytab as kudu/127.18.72.253@KRBTEST.COM (short username kudu)
I20260811 07:43:23.558837 10547 webserver.cc:460] Webserver started at http://127.18.72.253:36133/ using document root <none> and password file <none>
I20260811 07:43:23.559376 10547 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:23.559437 10547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:23.559682 10547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:23.561731 10547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/data/instance:
uuid: "cd403d3191a749e4b45c45ddd6c2322f"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "b3c0d53e35958f425a23b50a2ebac607"
server_key_iv: "b4ea3fb4716a834ec2be8262c3567549"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.562311 10547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/wal/instance:
uuid: "cd403d3191a749e4b45c45ddd6c2322f"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "b3c0d53e35958f425a23b50a2ebac607"
server_key_iv: "b4ea3fb4716a834ec2be8262c3567549"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.565997 10547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:43:23.568392 10563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:23.569595 10547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:23.569733 10547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/wal
uuid: "cd403d3191a749e4b45c45ddd6c2322f"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "b3c0d53e35958f425a23b50a2ebac607"
server_key_iv: "b4ea3fb4716a834ec2be8262c3567549"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.569888 10547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:23.590018 10547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:23.592942 10547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:23.593154 10547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:23.600657 10547 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.253:32951
I20260811 07:43:23.600951 10617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.253:32951 every 8 connection(s)
I20260811 07:43:23.601702 10547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
I20260811 07:43:23.604553 10620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:23.608227 10620 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:23.610925 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 10547
I20260811 07:43:23.611066 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-1/wal/instance
I20260811 07:43:23.611397 18723 external_mini_cluster.cc:1428] Setting key 99eaff141fbfa56870099f200490ec2d
I20260811 07:43:23.615135 10539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } attempt: 3
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.252@KRBTEST.COM" created.
I20260811 07:43:23.625089 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.610001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.18.72.254:40237)
0811 07:43:23.610262 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.610267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.611120 (+ 853us) server_negotiation.cc:407] Connection header received
0811 07:43:23.612523 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.612526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.612577 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.612649 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.614118 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.614990 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.616748 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.617119 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.617243 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.620264 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.620269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.620275 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.620309 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.621868 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.622534 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.622537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.622539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.622621 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.623202 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.623205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.623206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.623401 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.623513 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.624740 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.624932 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 07:43:23.625408 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.610456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:40237 (local address 127.0.0.1:46676)
0811 07:43:23.610949 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.611011 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.612257 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.612855 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.612866 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.613375 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.613970 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.613984 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.615147 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.615154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.616617 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.616627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.617407 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.617411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.617553 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.618134 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.618150 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.620066 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.622020 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.622030 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.622044 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.622440 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.622706 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.622708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.622710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.622892 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.623514 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.623521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.623771 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.624026 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.624354 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":63,"spinlock_wait_cycles":3840,"thread_start_us":93,"threads_started":1}
I20260811 07:43:23.625669 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.615545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43396)
0811 07:43:23.615694 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.615709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.615791 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.616505 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.616511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.616743 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.616914 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.616925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.618490 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.618493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.619190 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.619200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.619838 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.619842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.619984 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.620471 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.620486 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.622037 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.623847 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.623851 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.623853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.624194 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.624552 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.624556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.624558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.624651 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.625197 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.625202 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.625298 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.625382 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.625522 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 07:43:23.626335 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.615750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43396 (local address 127.18.72.253:32951)
0811 07:43:23.616066 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.616077 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.616099 (+ 22us) server_negotiation.cc:407] Connection header received
0811 07:43:23.616141 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.616146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.616325 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.616490 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.617074 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.618354 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.619313 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.619720 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.619837 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.622174 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.622182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.622195 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.622218 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.623742 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.624317 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.624325 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.624328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.624431 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.624754 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.624757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.624759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.625058 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.625863 (+ 805us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.625969 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.626119 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":220,"spinlock_wait_cycles":4352,"thread_start_us":93,"threads_started":1}
I20260811 07:43:23.627053 10620 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:23.628688 10539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:23.629424 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.629115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49418)
0811 07:43:23.629287 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.629345 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 07:43:23.630192 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.629927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49430)
0811 07:43:23.630082 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.630133 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 07:43:23.630333 10539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46497: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
I20260811 07:43:23.635583 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.627591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43400)
0811 07:43:23.627712 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.627724 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.627809 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.628076 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.628078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.628232 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.628317 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.628322 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.629154 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.629157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.629755 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.629768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.630466 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.630469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.630556 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.631025 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.631036 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.632215 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.634171 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.634174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.634175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.634363 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.634663 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.634665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.634667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.634736 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.635095 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.635096 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.635172 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.635250 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.635402 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 07:43:23.635598 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.627618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.18.72.253:32951)
0811 07:43:23.627746 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.627752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.627766 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:43:23.627941 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.627944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.627987 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.628065 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.628423 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.629046 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.629892 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.630239 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.630360 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.632351 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.632355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.632358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.632382 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.634057 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.634462 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.634467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.634471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.634569 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.634830 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.634833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.634835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.635015 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.635092 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.635319 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.635464 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 07:43:23.636484 10620 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:23.637151 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.636932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49432)
0811 07:43:23.637022 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.637093 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 07:43:23.637701 10567 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
I20260811 07:43:23.638531 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.638294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49438)
0811 07:43:23.638440 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.638488 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 07:43:23.638667 10620 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46497: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab.
Entry for principal kudu/127.18.72.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab.
I20260811 07:43:23.642800 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.252:46497
--webserver_interface=127.18.72.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab
--principal=kudu/127.18.72.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:33129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:40237,127.18.72.253:32951,127.18.72.252:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:43:23.661489 10539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } attempt: 1
I20260811 07:43:23.662302 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.662025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49444)
0811 07:43:23.662171 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.662229 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 07:43:23.662999 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.662773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49458)
0811 07:43:23.662902 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.662948 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 07:43:23.663125 10539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46497: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
I20260811 07:43:23.687964 10620 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } attempt: 1
I20260811 07:43:23.688751 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.688477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49460)
0811 07:43:23.688620 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.688679 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 07:43:23.689476 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.689255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49468)
0811 07:43:23.689367 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.689422 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 07:43:23.689608 10620 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46497: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
I20260811 07:43:23.738271 10539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } attempt: 2
I20260811 07:43:23.739128 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.738834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49474)
0811 07:43:23.738987 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.739052 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 07:43:23.739832 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.739621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49488)
0811 07:43:23.739735 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.739783 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 07:43:23.739991 10539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46497: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
I20260811 07:43:23.757804 10620 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } attempt: 2
I20260811 07:43:23.758570 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.758304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49490)
0811 07:43:23.758434 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.758499 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 07:43:23.758515 10627 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:43:23.758854 10627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:23.758905 10627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:23.758955 10627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 07:43:23.759361 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.759146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49492)
0811 07:43:23.759248 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.759298 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 07:43:23.759555 10620 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:46497: Network error: Client connection negotiation failed: client connection to 127.18.72.252:46497: connect: Connection refused (error 111)
W20260811 07:43:23.763011 10627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:23.763087 10627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:23.763137 10627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:23.763180 10627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:23.763231 10627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:23.768424 10627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:33129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:40237,127.18.72.253:32951,127.18.72.252:46497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab
--principal=kudu/127.18.72.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.252:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.72.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:23.769418 10627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:23.770407 10627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:23.778762 10632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:23.778889 10627 server_base.cc:1034] running on GCE node
W20260811 07:43:23.778759 10633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:23.778776 10636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:23.779736 10627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:23.780807 10627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:23.781991 10627 hybrid_clock.cc:648] HybridClock initialized: now 1786434203781960 us; error 42 us; skew 500 ppm
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:23.784658 10627 init.cc:377] Logged in from keytab as kudu/127.18.72.252@KRBTEST.COM (short username kudu)
I20260811 07:43:23.785480 10627 webserver.cc:460] Webserver started at http://127.18.72.252:33699/ using document root <none> and password file <none>
I20260811 07:43:23.786034 10627 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:23.786125 10627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:23.786355 10627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:23.788482 10627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/data/instance:
uuid: "92d45ae71f75480bbf4f143452a43e94"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "46d50fda897b93e990d0c2f40886a1db"
server_key_iv: "8ec35a4769fc323b8bfbd1969d562bfd"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.789031 10627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/wal/instance:
uuid: "92d45ae71f75480bbf4f143452a43e94"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "46d50fda897b93e990d0c2f40886a1db"
server_key_iv: "8ec35a4769fc323b8bfbd1969d562bfd"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.792578 10627 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:43:23.794848 10644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:23.796088 10627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:23.796221 10627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/wal
uuid: "92d45ae71f75480bbf4f143452a43e94"
format_stamp: "Formatted at 2026-08-11 07:43:23 on dist-test-slave-zkpd"
server_key: "46d50fda897b93e990d0c2f40886a1db"
server_key_iv: "8ec35a4769fc323b8bfbd1969d562bfd"
server_key_version: "encryptionkey@0"
I20260811 07:43:23.796324 10627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:23.816090 10627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:23.819180 10627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:23.819380 10627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:23.826330 10627 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.252:46497
I20260811 07:43:23.826623 10700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.252:46497 every 8 connection(s)
I20260811 07:43:23.827286 10627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
I20260811 07:43:23.828960 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 10627
I20260811 07:43:23.829075 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786433920963264-18723-0/minicluster-data/master-2/wal/instance
I20260811 07:43:23.829401 18723 external_mini_cluster.cc:1428] Setting key 6cff25f0a351b9c3bafae8de22ac8bf1
I20260811 07:43:23.830374 10701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:23.835137 10701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
I20260811 07:43:23.839186 10539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } attempt: 3
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
I20260811 07:43:23.846570 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.830799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.18.72.252:46497)
0811 07:43:23.831203 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.831223 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.831270 (+ 47us) server_negotiation.cc:407] Connection header received
0811 07:43:23.832161 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.832186 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.832590 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.832961 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.833675 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.835160 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.836327 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.836983 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.837154 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.839826 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.839832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.839843 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.839873 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.842144 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.843118 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.843148 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.843157 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.843293 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.843648 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.843651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.843652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.844191 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.844422 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.845192 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.845497 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
I20260811 07:43:23.848168 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.837020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.18.72.254:40237)
0811 07:43:23.837166 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.837171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.837518 (+ 347us) server_negotiation.cc:407] Connection header received
0811 07:43:23.837693 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.837699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.837750 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.837794 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.838534 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.839086 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.840418 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.840764 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.840878 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.843273 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.843277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.843279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.843302 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.845224 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.845817 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.845823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.845825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.845927 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.846247 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.846250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.846251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.846423 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.846526 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.847754 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.847984 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 07:43:23.848248 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.836985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:40237 (local address 127.0.0.1:46690)
0811 07:43:23.837403 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.837450 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.837567 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.837945 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.837949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.838263 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.838398 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.838404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.839248 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.839251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.840275 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.840286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.840885 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.840891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.841008 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.841529 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.841542 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.843127 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.845328 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.845336 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.845340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.845663 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.846044 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.846049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.846051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.846143 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.846529 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.846538 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.846686 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.847882 (+ 1196us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.848075 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":127,"threads_started":1}
I20260811 07:43:23.849421 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.839617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49524)
0811 07:43:23.839801 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.839814 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.839934 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.840698 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.840704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.840913 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.841019 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.841026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.841979 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.841985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.842612 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.842622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.843211 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.843217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.843300 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.843757 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.843774 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.845299 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.847702 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.847706 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.847708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.848021 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.848380 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.848383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.848385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.848472 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.848918 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.848922 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.849021 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.849125 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.849256 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 07:43:23.849457 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.839652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.18.72.252:46497)
0811 07:43:23.840459 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.840474 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.840487 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:43:23.840518 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.840521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.840559 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.840641 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.841143 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.841853 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.842737 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.843078 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.843189 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.845417 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.845421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.845423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.845445 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.847585 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.848136 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.848140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.848141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.848232 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.848599 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.848602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.848603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.848831 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.848928 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.849189 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.849294 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":78,"threads_started":1}
I20260811 07:43:23.849687 10701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:23.851432 10620 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } attempt: 3
I20260811 07:43:23.852944 10539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382: Bootstrap starting.
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
I20260811 07:43:23.857386 10539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:23.858354 10539 log.cc:824] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:23.859865 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.850192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:32951 (local address 127.0.0.1:43402)
0811 07:43:23.850423 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.850437 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.850500 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.850833 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.850836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.850996 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.851075 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.851081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.852737 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.852740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.853524 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.853532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.854244 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.854247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.854341 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.854761 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.854774 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.856145 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.858046 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.858051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.858054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.858309 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.858669 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.858675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.858680 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.858775 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.859356 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.859360 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.859445 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.859543 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.859675 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":1}
I20260811 07:43:23.860383 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.850471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43402 (local address 127.18.72.253:32951)
0811 07:43:23.850597 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.850604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.850620 (+ 16us) server_negotiation.cc:407] Connection header received
0811 07:43:23.850668 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.850671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.850713 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.850820 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.851193 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.852626 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.853646 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.854014 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.854097 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.856297 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.856301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.856304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.856326 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.857931 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.858438 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.858443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.858445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.858539 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.858902 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.858908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.858913 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.859217 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.859309 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.860115 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.860234 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 07:43:23.860387 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.849536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.18.72.254:40237)
0811 07:43:23.849630 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.849636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.849669 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:43:23.849804 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.849807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.849843 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.849930 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.850653 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.851308 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.852382 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.852679 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.852775 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.855647 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.855651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.855653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.855677 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.858136 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.858700 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.858706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.858711 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.858808 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.859157 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.859160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.859162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.859383 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.859541 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.860142 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.860250 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 07:43:23.861023 10701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:23.861302 10539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382: No bootstrap required, opened a new log
I20260811 07:43:23.861372 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.852113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49536)
0811 07:43:23.852342 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.852357 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.852418 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.852852 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.852855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.853036 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.853123 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.853129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.854132 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.854135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.854940 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.854952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.855622 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.855626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.855712 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.856179 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.856198 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.857689 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.859734 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.859737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.859739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.860029 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.860383 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.860386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.860388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.860480 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.860910 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.860913 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.861009 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.861091 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.861212 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 07:43:23.861837 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.852530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.18.72.252:46497)
0811 07:43:23.852639 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.852643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.852656 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:43:23.852693 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.852696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.852736 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.853227 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.853312 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.854015 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.855086 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.855472 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.855953 (+ 481us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.857847 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.857850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.857854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.857885 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.859612 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.860154 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.860158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.860161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.860242 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.860602 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.860605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.860607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.860800 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.861519 (+ 719us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.861599 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.861713 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:43:23.864769 10620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f: Bootstrap starting.
I20260811 07:43:23.866287 10539 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } }
I20260811 07:43:23.866525 10539 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2d96a794eac45f1a108e1a190843382, State: Initialized, Role: FOLLOWER
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
I20260811 07:43:23.866855 10539 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } }
I20260811 07:43:23.867877 10620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:23.868273 10539 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:23.868815 10620 log.cc:824] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f: Log is configured to *not* fsync() on all Append() calls
Aug 11 07:43:23 dist-test-slave-zkpd krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
W20260811 07:43:23.869059 10485 tablet.cc:2369] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:43:23.870994 10620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f: No bootstrap required, opened a new log
I20260811 07:43:23.871274 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.861466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:46497 (local address 127.0.0.1:49538)
0811 07:43:23.861840 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:23.861865 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:23.861944 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:23.862293 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:23.862295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:23.862466 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:23.862551 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.862558 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.863607 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.863610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.864443 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:23.864453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.865079 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.865082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:23.865152 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.865580 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:23.865593 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:23.866965 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:23.869655 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.869657 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.869659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.869907 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.870227 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:23.870230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:23.870231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:23.870320 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:23.870799 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:23.870802 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:23.870903 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 07:43:23.870991 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 07:43:23.871116 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304}
I20260811 07:43:23.871793 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.861512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49538 (local address 127.18.72.252:46497)
0811 07:43:23.861715 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.861718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.862119 (+ 401us) server_negotiation.cc:407] Connection header received
0811 07:43:23.862154 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.862156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.862193 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.862650 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.862725 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.863491 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.864573 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.864969 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.865321 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.867112 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.867117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.867122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.867150 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.868930 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.870014 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.870019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.870023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.870124 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.870443 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.870448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.870450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.870688 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.871442 (+ 754us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.871520 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.871625 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260811 07:43:23.873814 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.862538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.18.72.253:32951)
0811 07:43:23.862625 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.862629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.862739 (+ 110us) server_negotiation.cc:407] Connection header received
0811 07:43:23.862914 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.862917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.862956 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.863002 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.863781 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.864518 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.865514 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.865861 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.865975 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.869683 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.869688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.869693 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.869721 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.871550 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.872400 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.872406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.872411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.872498 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.872819 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.872823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.872825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.873032 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.873163 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.873542 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.873649 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 07:43:23.875466 10620 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } }
I20260811 07:43:23.875726 10620 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd403d3191a749e4b45c45ddd6c2322f, State: Initialized, Role: FOLLOWER
I20260811 07:43:23.876057 10620 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } }
I20260811 07:43:23.876151 10708 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.876257 10708 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 07:43:23.876284 10720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:43:23.876358 10720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:43:23.873612 10463 master_runner.cc:186] Error getting master registration for 127.18.72.253:32951: OK, instance_id { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" instance_seqno: 1786434203598616 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:43:23.876730 10463 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:43:23.877153 10620 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:23.880051 10701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94: Bootstrap starting.
I20260811 07:43:23.882640 10701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:23.883505 10701 log.cc:824] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:23.879410 10722 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.884258 10722 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:23.885509 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:43:23.887324 10508 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b2d96a794eac45f1a108e1a190843382"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46700
I20260811 07:43:23.887535 10508 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 07:43:23.887633 10508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:23.888967 10508 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } }
I20260811 07:43:23.889869 10508 leader_election.cc:288] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [CANDIDATE]: Term 1 election: Requested vote from peers cd403d3191a749e4b45c45ddd6c2322f (127.18.72.253:32951), 92d45ae71f75480bbf4f143452a43e94 (127.18.72.252:46497)
I20260811 07:43:23.890980 10591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2d96a794eac45f1a108e1a190843382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cd403d3191a749e4b45c45ddd6c2322f"
I20260811 07:43:23.891213 10591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:23.888978 10547 master_runner.cc:186] Error getting master registration for 127.18.72.252:46497: OK, instance_id { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" instance_seqno: 1786434203824432 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:43:23.892336 10547 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:43:23.892969 10591 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2d96a794eac45f1a108e1a190843382 in term 1.
I20260811 07:43:23.893412 10483 leader_election.cc:302] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2d96a794eac45f1a108e1a190843382, cd403d3191a749e4b45c45ddd6c2322f; no voters:
I20260811 07:43:23.893690 10708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:23.893940 10708 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [term 1 LEADER]: Becoming Leader. State: Replica: b2d96a794eac45f1a108e1a190843382, State: Running, Role: LEADER
I20260811 07:43:23.894264 10708 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } }
W20260811 07:43:23.882320 10733 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:43:23.895174 10733 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:43:23.896057 10708 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2d96a794eac45f1a108e1a190843382. Latest consensus state: current_term: 1 leader_uuid: "b2d96a794eac45f1a108e1a190843382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.896157 10708 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:23.897398 10701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94: No bootstrap required, opened a new log
I20260811 07:43:23.902278 10744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:23.903324 10744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:23.903509 10701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } }
I20260811 07:43:23.903851 10701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d45ae71f75480bbf4f143452a43e94, State: Initialized, Role: FOLLOWER
I20260811 07:43:23.904383 10701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } }
I20260811 07:43:23.906236 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2d96a794eac45f1a108e1a190843382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "92d45ae71f75480bbf4f143452a43e94"
I20260811 07:43:23.906461 10673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:23.908231 10673 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2d96a794eac45f1a108e1a190843382 in term 1.
I20260811 07:43:23.909411 10591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [term 1 FOLLOWER]: Refusing update from remote peer b2d96a794eac45f1a108e1a190843382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:43:23.910224 10738 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:43:23.914410 10722 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2d96a794eac45f1a108e1a190843382. Latest consensus state: current_term: 1 leader_uuid: "b2d96a794eac45f1a108e1a190843382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.914531 10722 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:23.914983 10673 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [term 1 FOLLOWER]: Refusing update from remote peer b2d96a794eac45f1a108e1a190843382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:43:23.915470 10738 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:43:23.918874 10701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:23.922372 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.900326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.18.72.253:32951)
0811 07:43:23.900438 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.900443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.908983 (+ 8540us) server_negotiation.cc:407] Connection header received
0811 07:43:23.909476 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.909479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.909523 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.909576 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.910646 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.911375 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.913420 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.913789 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.913861 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.917742 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.917749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.917753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.917779 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.919488 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.919996 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.920000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.920002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.920086 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.920405 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.920408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.920410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.920621 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.920731 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.922090 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.922207 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:43:23.926123 10738 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2d96a794eac45f1a108e1a190843382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.926237 10738 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:23.926342 10749 mvcc.cc:205] Tried to move back new op lower bound from 7317234499203747840 to 7317234499154362368. Current Snapshot: MvccSnapshot[applied={T|T < 7317234499203747840}]
I20260811 07:43:23.929289 10708 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2d96a794eac45f1a108e1a190843382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.929396 10708 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:23.930202 10722 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2d96a794eac45f1a108e1a190843382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.930310 10722 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:23.932742 10753 mvcc.cc:205] Tried to move back new op lower bound from 7317234499203747840 to 7317234499154362368. Current Snapshot: MvccSnapshot[applied={T|T < 7317234499203747840}]
I20260811 07:43:23.934406 10750 mvcc.cc:205] Tried to move back new op lower bound from 7317234499203747840 to 7317234499154362368. Current Snapshot: MvccSnapshot[applied={T|T < 7317234499203747840}]
I20260811 07:43:23.934633 10752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2d96a794eac45f1a108e1a190843382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.934753 10752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:23.934832 10752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2d96a794eac45f1a108e1a190843382. Latest consensus state: current_term: 1 leader_uuid: "b2d96a794eac45f1a108e1a190843382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.934913 10752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:23.934979 10752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2d96a794eac45f1a108e1a190843382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2d96a794eac45f1a108e1a190843382" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 40237 } } peers { permanent_uuid: "cd403d3191a749e4b45c45ddd6c2322f" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 32951 } } peers { permanent_uuid: "92d45ae71f75480bbf4f143452a43e94" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 46497 } } }
I20260811 07:43:23.935045 10752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:23.936851 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.900323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.18.72.254:40237)
0811 07:43:23.900438 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.900443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.908155 (+ 7712us) server_negotiation.cc:407] Connection header received
0811 07:43:23.908385 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.908391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.908444 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.908497 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.910482 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.911265 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.921027 (+ 9762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.921379 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.921451 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.929728 (+ 8277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.929732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.929735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.929761 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.931474 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.934364 (+ 2890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.934368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.934370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.934454 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.935178 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.935182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.935184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.935412 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.935527 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.936586 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.936704 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 07:43:23.938468 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:23.901214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.18.72.252:46497)
0811 07:43:23.901336 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:23.901343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:23.909216 (+ 7873us) server_negotiation.cc:407] Connection header received
0811 07:43:23.909342 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:23.909346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:23.909396 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:23.909470 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:23.915841 (+ 6371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.916611 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.923753 (+ 7142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:23.924120 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:23.924194 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:23.932034 (+ 7840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:23.932038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:23.932041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:23.932066 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:23.933857 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.934761 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.934766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.934768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.934857 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:23.935833 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:23.935836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:23.935838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:23.936103 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:23.936222 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:23.938183 (+ 1961us) server_negotiation.cc:299] Negotiation successful
0811 07:43:23.938315 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 07:43:23.939567 10767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:43:23.939646 10767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:43:23.942652 10744 catalog_manager.cc:1347] Generated new cluster ID: 936883b31be64a4f8b887c5fff5fbe97
I20260811 07:43:23.942718 10744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:23.960983 10744 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:23.962220 10744 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:23.977419 10744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2d96a794eac45f1a108e1a190843382: Generated new TSK 0
I20260811 07:43:23.978216 10744 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:43:24.008896 10704 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.005768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.18.72.252:46497)
0811 07:43:24.005879 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.005883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.007602 (+ 1719us) server_negotiation.cc:407] Connection header received
0811 07:43:24.008088 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.008092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.008148 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.008199 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.008757 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49558: BlockingRecv error: recv got EOF from 127.0.0.1:49558 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 07:43:24.009467 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.005056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.18.72.253:32951)
0811 07:43:24.005178 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.005182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.007025 (+ 1843us) server_negotiation.cc:407] Connection header received
0811 07:43:24.007214 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.007218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.007266 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.007331 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.009354 (+ 2023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43416: BlockingRecv error: recv got EOF from 127.0.0.1:43416 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 07:43:24.010962 10542 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.006444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46718 (local address 127.18.72.254:40237)
0811 07:43:24.006552 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.006556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.009933 (+ 3377us) server_negotiation.cc:407] Connection header received
0811 07:43:24.010275 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.010279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.010340 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.010392 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.010852 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46718: BlockingRecv error: recv got EOF from 127.0.0.1:46718 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 07:43:24.015259 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.013770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.18.72.253:32951)
0811 07:43:24.013880 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.013884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.014381 (+ 497us) server_negotiation.cc:407] Connection header received
0811 07:43:24.014580 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.014583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.014648 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.014695 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.015171 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43420: BlockingRecv error: recv got EOF from 127.0.0.1:43420 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 07:43:24.019840 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.018410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.18.72.253:32951)
0811 07:43:24.018497 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.018501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.019003 (+ 502us) server_negotiation.cc:407] Connection header received
0811 07:43:24.019175 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.019178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.019221 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.019270 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.019738 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43432: BlockingRecv error: recv got EOF from 127.0.0.1:43432 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 07:43:24.025197 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.023647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.18.72.253:32951)
0811 07:43:24.023739 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.023742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.024366 (+ 624us) server_negotiation.cc:407] Connection header received
0811 07:43:24.024541 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.024544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.024590 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.024638 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.025110 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43446: BlockingRecv error: recv got EOF from 127.0.0.1:43446 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 07:43:24.032649 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.031066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43450 (local address 127.18.72.253:32951)
0811 07:43:24.031192 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.031196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.031727 (+ 531us) server_negotiation.cc:407] Connection header received
0811 07:43:24.031930 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.031934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.031986 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.032034 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.032553 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43450: BlockingRecv error: recv got EOF from 127.0.0.1:43450 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 07:43:24.044363 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.042661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.18.72.253:32951)
0811 07:43:24.042765 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.042770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.043358 (+ 588us) server_negotiation.cc:407] Connection header received
0811 07:43:24.043554 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.043558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.043626 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.043685 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.044247 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43456: BlockingRecv error: recv got EOF from 127.0.0.1:43456 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 07:43:24.064179 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.062505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.18.72.253:32951)
0811 07:43:24.062618 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.062622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.063177 (+ 555us) server_negotiation.cc:407] Connection header received
0811 07:43:24.063382 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.063386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.063435 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.063484 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.064066 (+ 582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43464: BlockingRecv error: recv got EOF from 127.0.0.1:43464 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 07:43:24.099815 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.098480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43468 (local address 127.18.72.253:32951)
0811 07:43:24.098613 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.098621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.098878 (+ 257us) server_negotiation.cc:407] Connection header received
0811 07:43:24.098976 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.098980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.099045 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.099144 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.099694 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43468: BlockingRecv error: recv got EOF from 127.0.0.1:43468 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 07:43:24.167092 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.165982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43478 (local address 127.18.72.253:32951)
0811 07:43:24.166157 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.166161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.166224 (+ 63us) server_negotiation.cc:407] Connection header received
0811 07:43:24.166361 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.166363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.166412 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.166485 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.166942 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43478: BlockingRecv error: recv got EOF from 127.0.0.1:43478 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 07:43:24.298272 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.297190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43486 (local address 127.18.72.253:32951)
0811 07:43:24.297332 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.297336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.297496 (+ 160us) server_negotiation.cc:407] Connection header received
0811 07:43:24.297625 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.297628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.297678 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.297754 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.298175 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43486: BlockingRecv error: recv got EOF from 127.0.0.1:43486 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 07:43:24.557617 10624 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:24.556581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43494 (local address 127.18.72.253:32951)
0811 07:43:24.556728 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:24.556732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:24.556750 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:43:24.556905 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:24.556908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:24.556955 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:24.557026 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:24.557473 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43494: BlockingRecv error: recv got EOF from 127.0.0.1:43494 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 07:43:24.896652 10733 catalog_manager.cc:1259] Loaded cluster ID: 936883b31be64a4f8b887c5fff5fbe97
I20260811 07:43:24.896736 10733 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f: loading cluster ID for follower catalog manager: success
I20260811 07:43:24.898424 10733 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f: acquiring CA information for follower catalog manager: success
I20260811 07:43:24.899659 10733 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd403d3191a749e4b45c45ddd6c2322f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:43:24.941401 10767 catalog_manager.cc:1259] Loaded cluster ID: 936883b31be64a4f8b887c5fff5fbe97
I20260811 07:43:24.941506 10767 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94: loading cluster ID for follower catalog manager: success
I20260811 07:43:24.943184 10767 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94: acquiring CA information for follower catalog manager: success
I20260811 07:43:24.944454 10767 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92d45ae71f75480bbf4f143452a43e94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:43:25.075911 10847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:25.071785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.18.72.253:32951)
0811 07:43:25.072073 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:25.072077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:25.072094 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:43:25.072207 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:25.072209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:25.072343 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:25.072441 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:25.072746 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.073388 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.074400 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.074623 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.074703 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:25.075007 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:25.075451 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:25.075553 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:25.075624 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 07:43:25.075669 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":118,"threads_started":1}
I20260811 07:43:25.082827 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:25.079256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.18.72.252:46497)
0811 07:43:25.079523 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:25.079527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:25.079540 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:43:25.079623 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:25.079625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:25.079738 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:25.079841 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:25.080101 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.080615 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.081369 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.081582 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.081658 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:25.081960 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:25.082401 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:25.082500 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:25.082594 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 07:43:25.082638 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260811 07:43:25.090212 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:25.086436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.18.72.252:46497)
0811 07:43:25.086738 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:25.086741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:25.086771 (+ 30us) server_negotiation.cc:407] Connection header received
0811 07:43:25.086889 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:25.086892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:25.086949 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:25.087024 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:25.087263 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.088021 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.089066 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.089367 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.089477 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:25.089741 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:25.089819 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:25.089894 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:25.090022 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 07:43:25.090075 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238}
I20260811 07:43:25.090704 10860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:25.085783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46730 (local address 127.18.72.254:40237)
0811 07:43:25.086039 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:25.086043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:25.086138 (+ 95us) server_negotiation.cc:407] Connection header received
0811 07:43:25.086331 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:25.086335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:25.086497 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:25.086641 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:25.087096 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.087681 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.088860 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.089138 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.089234 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:25.089563 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:25.089866 (+ 303us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:25.089954 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:25.090495 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 07:43:25.090556 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260811 07:43:25.091202 10847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:25.085780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.18.72.253:32951)
0811 07:43:25.085894 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:25.085898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:25.086320 (+ 422us) server_negotiation.cc:407] Connection header received
0811 07:43:25.086494 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:25.086497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:25.086553 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:25.086604 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:25.087200 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.088028 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.089990 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:25.090325 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:25.090400 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:25.090757 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:25.090839 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:25.090905 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:25.091025 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 07:43:25.091073 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:43:25.093940 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 10463
I20260811 07:43:25.103045 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 10547
I20260811 07:43:25.111171 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 10627
2026-08-11T07:43:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3132 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:43:25 dist-test-slave-zkpd krb5kdc[10865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:43:25 dist-test-slave-zkpd krb5kdc[10865](info): set up 2 sockets
Aug 11 07:43:25 dist-test-slave-zkpd krb5kdc[10865](info): commencing operation
krb5kdc: starting...
W20260811 07:43:26.189199 18723 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/test-user.keytab.
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:43:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:43:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.72.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
Entry for principal kudu/127.18.72.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab.
I20260811 07:43:26.314683 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:38815
--webserver_interface=127.18.72.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:38815,127.18.72.253:43999,127.18.72.252:42479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:43:26.426021 10878 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:43:26.426282 10878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:26.426306 10878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:26.426342 10878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:26.430343 10878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:26.430393 10878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:26.430430 10878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:26.430447 10878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:26.430462 10878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:26.435663 10878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:38815,127.18.72.253:43999,127.18.72.252:42479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.254.keytab
--principal=kudu/127.18.72.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.254:38815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.72.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:26.436674 10878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:26.437700 10878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:26.446046 10884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:26.446041 10883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:26.446090 10878 server_base.cc:1034] running on GCE node
W20260811 07:43:26.446036 10886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:26.446794 10878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:26.447781 10878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:26.448944 10878 hybrid_clock.cc:648] HybridClock initialized: now 1786434206448926 us; error 29 us; skew 500 ppm
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:26.451596 10878 init.cc:377] Logged in from keytab as kudu/127.18.72.254@KRBTEST.COM (short username kudu)
I20260811 07:43:26.452482 10878 webserver.cc:460] Webserver started at http://127.18.72.254:38685/ using document root <none> and password file <none>
I20260811 07:43:26.453027 10878 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:26.453116 10878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:26.453341 10878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:26.455317 10878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/data/instance:
uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "81c342bffd91411aaf4281f15546c057"
server_key_iv: "03f293a5451ccd6500e153a5afd04773"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.455940 10878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/wal/instance:
uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "81c342bffd91411aaf4281f15546c057"
server_key_iv: "03f293a5451ccd6500e153a5afd04773"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.459514 10878 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:43:26.461875 10893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:26.463068 10878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:43:26.463205 10878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/wal
uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "81c342bffd91411aaf4281f15546c057"
server_key_iv: "03f293a5451ccd6500e153a5afd04773"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.463310 10878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:26.481247 10878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:26.483947 10878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:26.484143 10878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:26.491191 10878 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.254:38815
I20260811 07:43:26.491489 10949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.254:38815 every 8 connection(s)
I20260811 07:43:26.492544 10878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/data/info.pb
I20260811 07:43:26.495319 10950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:26.499008 10950 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:26.501706 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 10878
I20260811 07:43:26.501861 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-0/wal/instance
I20260811 07:43:26.502159 18723 external_mini_cluster.cc:1428] Setting key abe96895d7bb6b308568abdb7f6cea7d
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.72.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.253@KRBTEST.COM" created.
I20260811 07:43:26.514890 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.501101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.18.72.254:38815)
0811 07:43:26.501520 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.501542 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.501816 (+ 274us) server_negotiation.cc:407] Connection header received
0811 07:43:26.502865 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.502874 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.503129 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.503333 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.504117 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.505165 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.506646 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.506994 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.507107 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.509908 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.509913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.509923 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.509944 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.511644 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.512241 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.512247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.512248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.512360 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.512711 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.512714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.512716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.513158 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.513305 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.513686 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.514003 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":277,"thread_start_us":138,"threads_started":1}
I20260811 07:43:26.514923 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.501101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:38815 (local address 127.0.0.1:60896)
0811 07:43:26.501638 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.501686 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.502547 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.503341 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.503346 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.503670 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.503964 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.503976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.505300 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.505305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.506521 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.506536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.507166 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.507175 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.507363 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.507980 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.508005 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.509729 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.511784 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.511795 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.511805 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.512124 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.512482 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.512485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.512487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.512609 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.513280 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.513287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.513437 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.513604 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.514122 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":155,"spinlock_wait_cycles":39296,"thread_start_us":161,"threads_started":1}
I20260811 07:43:26.517525 10950 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:26.518287 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.518058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47030)
0811 07:43:26.518170 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.518236 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 07:43:26.518492 10895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
I20260811 07:43:26.519351 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.519153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47036)
0811 07:43:26.519255 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.519304 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 07:43:26.519552 10950 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:43999: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab.
Entry for principal kudu/127.18.72.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab.
I20260811 07:43:26.531525 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.253:43999
--webserver_interface=127.18.72.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab
--principal=kudu/127.18.72.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:38815,127.18.72.253:43999,127.18.72.252:42479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:43:26.568759 10950 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } attempt: 1
I20260811 07:43:26.569657 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.569353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47040)
0811 07:43:26.569507 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.569578 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 07:43:26.570372 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.570163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47046)
0811 07:43:26.570268 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.570319 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 07:43:26.570508 10950 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:43999: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
I20260811 07:43:26.638682 10950 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } attempt: 2
I20260811 07:43:26.639504 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.639216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47060)
0811 07:43:26.639366 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.639432 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 07:43:26.640268 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.640038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47074)
0811 07:43:26.640146 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.640216 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 07:43:26.640403 10950 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.253:43999: Network error: Client connection negotiation failed: client connection to 127.18.72.253:43999: connect: Connection refused (error 111)
W20260811 07:43:26.643344 10957 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:43:26.643633 10957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:26.643694 10957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:26.643770 10957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:26.647751 10957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:26.647822 10957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:26.647873 10957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:26.647931 10957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:26.647967 10957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:26.653040 10957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:38815,127.18.72.253:43999,127.18.72.252:42479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.253.keytab
--principal=kudu/127.18.72.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.253:43999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.72.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:26.654031 10957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:26.655006 10957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:26.663542 10963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:26.663576 10966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:26.663559 10964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:26.663828 10957 server_base.cc:1034] running on GCE node
I20260811 07:43:26.664454 10957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:26.665534 10957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:26.666775 10957 hybrid_clock.cc:648] HybridClock initialized: now 1786434206666746 us; error 29 us; skew 500 ppm
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:26.669560 10957 init.cc:377] Logged in from keytab as kudu/127.18.72.253@KRBTEST.COM (short username kudu)
I20260811 07:43:26.670400 10957 webserver.cc:460] Webserver started at http://127.18.72.253:43709/ using document root <none> and password file <none>
I20260811 07:43:26.671000 10957 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:26.671095 10957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:26.671324 10957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:26.673373 10957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/data/instance:
uuid: "89774c15dd6343c3bb0c880aa2370c68"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "9cd1da9ac6c48a413c2722f48dda0f11"
server_key_iv: "615c02ec7f2fe1dfa9fdc0b9d6f20b22"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.673952 10957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/wal/instance:
uuid: "89774c15dd6343c3bb0c880aa2370c68"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "9cd1da9ac6c48a413c2722f48dda0f11"
server_key_iv: "615c02ec7f2fe1dfa9fdc0b9d6f20b22"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.677634 10957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:43:26.680073 10974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:26.681275 10957 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 07:43:26.681414 10957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/wal
uuid: "89774c15dd6343c3bb0c880aa2370c68"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "9cd1da9ac6c48a413c2722f48dda0f11"
server_key_iv: "615c02ec7f2fe1dfa9fdc0b9d6f20b22"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.681522 10957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:26.694597 10957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:26.697284 10957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:26.697481 10957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:26.704094 10957 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.253:43999
I20260811 07:43:26.704394 11033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.253:43999 every 8 connection(s)
I20260811 07:43:26.705025 10957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/data/info.pb
I20260811 07:43:26.707664 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 10957
I20260811 07:43:26.707814 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-1/wal/instance
I20260811 07:43:26.707878 11034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:26.708192 18723 external_mini_cluster.cc:1428] Setting key b6fbf0b0eceea06b160d08dea7f0253b
I20260811 07:43:26.712239 11034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.72.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.72.252@KRBTEST.COM" created.
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:43:26.726218 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.714160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60898 (local address 127.18.72.254:38815)
0811 07:43:26.714321 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.714326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.715122 (+ 796us) server_negotiation.cc:407] Connection header received
0811 07:43:26.716079 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.716085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.716157 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.716269 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.717358 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.718171 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.719276 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.719515 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.719608 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.722105 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.722110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.722115 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.722149 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.723733 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.724372 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.724378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.724383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.724486 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.724849 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.724856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.724860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.725140 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.725255 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.725798 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.725956 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 07:43:26.726747 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.714490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:38815 (local address 127.0.0.1:60898)
0811 07:43:26.714939 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.714990 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.715815 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.716434 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.716441 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.716766 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.717207 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.717220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.718296 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.718298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.719163 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.719172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.719682 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.719686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.719855 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.720495 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.720539 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.721930 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.723936 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.723942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.723953 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.724286 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.724592 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.724594 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.724595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.724719 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.725261 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.725265 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.725516 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.725664 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.725911 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":117,"threads_started":1}
I20260811 07:43:26.728338 11034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:26.731555 10950 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } attempt: 3
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
I20260811 07:43:26.739228 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.728893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47088)
0811 07:43:26.728989 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.729004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.729075 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.729607 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.729609 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.729786 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.729862 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.729867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.730879 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.730882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.731414 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.731421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.732221 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.732225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.732375 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.732913 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.732949 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.734676 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.737291 (+ 2615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.737298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.737302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.737542 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.737912 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.737916 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.737918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.738043 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.738567 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.738573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.738695 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.738819 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.739003 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 07:43:26.739535 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.728987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.18.72.253:43999)
0811 07:43:26.729285 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.729293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.729310 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:43:26.729342 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.729346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.729491 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.729632 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.729968 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.730754 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.731527 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.732084 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.732189 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.734862 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.734875 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.734896 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.734928 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.737164 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.737670 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.737677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.737679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.737768 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.738154 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.738157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.738158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.738462 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.738584 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.738906 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.739080 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":104,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.72.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab.
Entry for principal kudu/127.18.72.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab.
I20260811 07:43:26.740779 18723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
/tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.252:42479
--webserver_interface=127.18.72.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab
--principal=kudu/127.18.72.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.72.212:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.72.254:38815,127.18.72.253:43999,127.18.72.252:42479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:43:26.743201 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.732096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.18.72.253:43999)
0811 07:43:26.732684 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.732693 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.732705 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:43:26.732741 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.732745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.732789 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.732884 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.733563 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.734441 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.736280 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.736668 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.737201 (+ 533us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.739058 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.739062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.739065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.739087 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.741449 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.741883 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.741889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.741894 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.741987 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.742239 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.742245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.742249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.742478 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.742585 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.742939 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.743047 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":181,"threads_started":1}
I20260811 07:43:26.743237 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.732017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47092)
0811 07:43:26.732390 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.732411 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.732474 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.732921 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.732924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.733118 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.733242 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.733249 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.734564 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.734569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.736114 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.736125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.736850 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.736855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.736980 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.737457 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.737478 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.738939 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.741579 (+ 2640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.741581 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.741583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.741788 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.742079 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.742081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.742082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.742152 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.742587 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.742592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.742713 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.742867 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.743020 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271}
I20260811 07:43:26.744026 11034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:26.744369 10950 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:43:26.744818 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.744541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58864)
0811 07:43:26.744654 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.744752 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 07:43:26.745112 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.744799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58868)
0811 07:43:26.744946 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.745012 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 07:43:26.745100 10977 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
I20260811 07:43:26.746028 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.745784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58888)
0811 07:43:26.745922 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.745969 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 07:43:26.746105 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.745596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58874)
0811 07:43:26.745950 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.745995 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":98,"threads_started":1}
W20260811 07:43:26.746250 11034 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:42479: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
W20260811 07:43:26.746301 10950 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:42479: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
I20260811 07:43:26.777451 10950 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } attempt: 1
I20260811 07:43:26.778309 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.778021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58892)
0811 07:43:26.778174 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.778235 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 07:43:26.779052 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.778819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58904)
0811 07:43:26.778946 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.778996 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 07:43:26.779186 10950 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:42479: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
I20260811 07:43:26.795580 11034 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } attempt: 1
I20260811 07:43:26.796428 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.796145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58914)
0811 07:43:26.796291 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.796356 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 07:43:26.797170 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.796950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58922)
0811 07:43:26.797061 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.797117 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 07:43:26.797302 11034 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:42479: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
I20260811 07:43:26.854367 10950 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } attempt: 2
I20260811 07:43:26.855186 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.854896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58936)
0811 07:43:26.855048 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.855113 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 07:43:26.855986 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.855704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58944)
0811 07:43:26.855813 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.855874 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 07:43:26.856112 10950 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:42479: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
W20260811 07:43:26.856006 11041 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:43:26.856377 11041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:43:26.856443 11041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:43:26.856513 11041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:43:26.860579 11041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:43:26.860651 11041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:43:26.860695 11041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:43:26.860750 11041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:43:26.860788 11041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:43:26.865463 11034 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } attempt: 2
I20260811 07:43:26.866065 11041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.72.212:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.72.254:38815,127.18.72.253:43999,127.18.72.252:42479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/krb5kdc/kudu_127.18.72.252.keytab
--principal=kudu/127.18.72.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.72.252:42479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.72.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 06:46:31 UTC on ccycloud.quasar-atskvk.root.comops.site
build id 4579820
I20260811 07:43:26.866241 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.865891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58946)
0811 07:43:26.866018 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.866080 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 07:43:26.866998 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.866787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58960)
0811 07:43:26.866892 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.866939 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 07:43:26.867121 11034 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.72.252:42479: Network error: Client connection negotiation failed: client connection to 127.18.72.252:42479: connect: Connection refused (error 111)
I20260811 07:43:26.867118 11041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:43:26.868235 11041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:43:26.876408 11051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:26.876446 11048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:43:26.876431 11049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:43:26.876655 11041 server_base.cc:1034] running on GCE node
I20260811 07:43:26.877256 11041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:43:26.878316 11041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:43:26.879503 11041 hybrid_clock.cc:648] HybridClock initialized: now 1786434206879470 us; error 49 us; skew 500 ppm
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:43:26.882249 11041 init.cc:377] Logged in from keytab as kudu/127.18.72.252@KRBTEST.COM (short username kudu)
I20260811 07:43:26.883075 11041 webserver.cc:460] Webserver started at http://127.18.72.252:41733/ using document root <none> and password file <none>
I20260811 07:43:26.883641 11041 fs_manager.cc:359] Metadata directory not provided
I20260811 07:43:26.883731 11041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:43:26.884019 11041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:43:26.886121 11041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/data/instance:
uuid: "1d792be4c96949a7be56319eaeb99fb3"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "fd172181de55035d4fb8a571bffaaf7b"
server_key_iv: "790d93dca25819f5b0ac2e98fc238613"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.886729 11041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/wal/instance:
uuid: "1d792be4c96949a7be56319eaeb99fb3"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "fd172181de55035d4fb8a571bffaaf7b"
server_key_iv: "790d93dca25819f5b0ac2e98fc238613"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.890600 11041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 07:43:26.893203 11059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:43:26.894419 11041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:43:26.894572 11041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/data,/tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/wal
uuid: "1d792be4c96949a7be56319eaeb99fb3"
format_stamp: "Formatted at 2026-08-11 07:43:26 on dist-test-slave-zkpd"
server_key: "fd172181de55035d4fb8a571bffaaf7b"
server_key_iv: "790d93dca25819f5b0ac2e98fc238613"
server_key_version: "encryptionkey@0"
I20260811 07:43:26.894681 11041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:43:26.906893 11041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:43:26.909714 11041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:43:26.909929 11041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:43:26.916503 11041 rpc_server.cc:307] RPC server started. Bound to: 127.18.72.252:42479
I20260811 07:43:26.916802 11116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.72.252:42479 every 8 connection(s)
I20260811 07:43:26.917471 11041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/data/info.pb
I20260811 07:43:26.918929 18723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu as pid 11041
I20260811 07:43:26.919090 18723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskIvL2Zc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786433920963264-18723-0/minicluster-data/master-2/wal/instance
I20260811 07:43:26.919410 18723 external_mini_cluster.cc:1428] Setting key d73d0babf47f297765928f5b95d08551
I20260811 07:43:26.920620 11117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:43:26.925386 11117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
I20260811 07:43:26.935365 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.921152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.18.72.252:42479)
0811 07:43:26.921548 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.921567 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.921607 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:43:26.922512 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.922536 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.922962 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.923338 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.924062 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.925621 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.926664 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.927187 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.927397 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.930430 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.930447 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.930459 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.930492 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.932399 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.932893 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.932905 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.932912 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.933016 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.933305 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.933310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.933311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.933757 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.933956 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.934270 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.934575 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":112,"threads_started":1}
I20260811 07:43:26.936669 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.927323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.18.72.254:38815)
0811 07:43:26.927457 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.927461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.927881 (+ 420us) server_negotiation.cc:407] Connection header received
0811 07:43:26.928092 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.928097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.928152 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.928225 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.928902 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.929563 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.930635 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.930842 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.930963 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.933209 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.933212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.933214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.933238 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.934797 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.935278 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.935291 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.935297 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.935372 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.935800 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.935803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.935804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.936042 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.936132 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.936465 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.936563 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:43:26.937448 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.927259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.254:38815 (local address 127.0.0.1:60900)
0811 07:43:26.927719 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.927776 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.927929 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.928289 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.928294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.928630 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.928758 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.928763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.929697 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.929701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.930511 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.930522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.930955 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.930961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.931083 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.931558 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.931577 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.933064 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.934908 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.934918 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.934924 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.935174 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.935597 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.935600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.935602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.935697 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.936146 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.936151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.936267 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.936415 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.937277 (+ 862us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"spinlock_wait_cycles":896,"thread_start_us":162,"threads_started":1}
I20260811 07:43:26.938668 11117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.254@KRBTEST.COM
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
I20260811 07:43:26.947333 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.939167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.253:43999 (local address 127.0.0.1:47106)
0811 07:43:26.939297 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.939310 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.939392 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.939643 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.939645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.939771 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.939872 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.939878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.940696 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.940700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.941428 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.941439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.942032 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.942036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.942179 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.942587 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.942601 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.944133 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.945808 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.945813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.945815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.946024 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.946318 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.946322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.946324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.946412 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.946792 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.946795 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.946907 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.947015 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.947156 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 07:43:26.947336 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.939296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.18.72.253:43999)
0811 07:43:26.939431 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.939437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.939449 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:43:26.939494 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.939496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.939542 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.939623 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.940038 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.940582 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.941573 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.941909 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.941993 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.944282 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.944285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.944287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.944315 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.945687 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.946125 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.946132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.946134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.946223 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.946503 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.946508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.946509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.946695 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.946802 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.947070 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.947201 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:43:26.947441 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.938150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.18.72.254:38815)
0811 07:43:26.938376 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.938382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.938396 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:43:26.938428 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.938431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.938473 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.938555 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.939310 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.940029 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.940826 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.941033 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.941110 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.943933 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.943936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.943938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.943961 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.945595 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.946033 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.946039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.946040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.946121 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.946420 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.946424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.946426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.946635 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.946932 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.947007 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.947125 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260811 07:43:26.948271 11117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.252@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.72.253@KRBTEST.COM
I20260811 07:43:26.955263 10950 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } attempt: 3
I20260811 07:43:26.957733 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.948840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58974)
0811 07:43:26.949161 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.949171 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.949253 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.949499 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.949504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.949689 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.949790 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.949797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.950812 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.950816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.951626 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.951635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.952239 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.952242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.952332 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.952777 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.952790 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.954174 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.956132 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.956135 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.956136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.956387 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.956654 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.956659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.956661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.956759 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.957191 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.957195 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.957314 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.957430 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.957564 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":3}
I20260811 07:43:26.957851 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.949046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58974 (local address 127.18.72.252:42479)
0811 07:43:26.949179 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.949185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.949217 (+ 32us) server_negotiation.cc:407] Connection header received
0811 07:43:26.949348 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.949351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.949394 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.949451 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.949901 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.950688 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.951744 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.952129 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.952220 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.954304 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.954307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.954308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.954329 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.956018 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.956481 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.956488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.956489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.956567 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.956843 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.956848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.956853 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.957080 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.957189 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.957512 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.957632 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:43:26.958256 11034 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } attempt: 3
I20260811 07:43:26.959282 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.948811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.18.72.253:43999)
0811 07:43:26.948944 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.948947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.948959 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:43:26.949092 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.949098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.949142 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.949223 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.950006 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.950723 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.951744 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.952108 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.952220 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.954966 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.954970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.954972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.954996 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.957134 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.957601 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.957604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.957608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.957688 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.957972 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.957976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.957978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.958224 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.958452 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.959042 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.959152 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.254@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
I20260811 07:43:26.961947 11117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3: Bootstrap starting.
I20260811 07:43:26.963120 18723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 11 07:43:26 dist-test-slave-zkpd krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786434206, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.72.253@KRBTEST.COM for kudu/127.18.72.252@KRBTEST.COM
I20260811 07:43:26.965373 11117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:26.964437 10923 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60916
I20260811 07:43:26.965946 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.955641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58978)
0811 07:43:26.955743 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.955756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.955836 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.956310 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.956314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.956532 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.956670 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.956678 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.958165 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.958168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.959011 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.959026 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.959941 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.959946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.960048 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.960515 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.960530 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.961993 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.964301 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.964304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.964306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.964554 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.964887 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.964890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.964892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.964978 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.965441 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.965447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.965544 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.965631 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.965796 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 07:43:26.966387 11117 log.cc:824] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:26.966629 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.955685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.18.72.252:42479)
0811 07:43:26.956053 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.956059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.956069 (+ 10us) server_negotiation.cc:407] Connection header received
0811 07:43:26.956121 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.956124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.956166 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.956260 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.956788 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.958025 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.959373 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.959751 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.959924 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.962141 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.962145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.962147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.962170 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.963875 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.964679 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.964682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.964684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.964763 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.965089 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.965092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.965094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.965309 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.966187 (+ 878us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.966267 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.966373 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":121,"threads_started":1}
I20260811 07:43:26.968663 11117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3: No bootstrap required, opened a new log
I20260811 07:43:26.969378 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.958597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.72.252:42479 (local address 127.0.0.1:58994)
0811 07:43:26.959182 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 07:43:26.959193 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 07:43:26.959246 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:43:26.960666 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:43:26.960670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:43:26.960836 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:43:26.960952 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.960961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.962339 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.962343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.963095 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:43:26.963105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.963683 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.963687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:43:26.963775 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.964270 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:43:26.964287 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:43:26.965715 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:43:26.967799 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.967805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.967807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.968076 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.968380 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:43:26.968383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:43:26.968388 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:43:26.968478 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:43:26.968931 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:43:26.968934 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:43:26.969027 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 07:43:26.969109 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 07:43:26.969237 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504}
I20260811 07:43:26.969835 10950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97: Bootstrap starting.
I20260811 07:43:26.969937 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:26.958800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58994 (local address 127.18.72.252:42479)
0811 07:43:26.960421 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:26.960429 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:26.960441 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:43:26.960476 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:26.960479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:26.960518 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:26.961410 (+ 892us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:26.961527 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.962212 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.963237 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:26.963543 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:26.964023 (+ 480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:26.965875 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:26.965879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:26.965881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:26.965907 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:26.967669 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.968199 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.968202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.968204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.968278 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:26.968600 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:26.968603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:26.968605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:26.968796 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:26.969550 (+ 754us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:26.969646 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 07:43:26.969756 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1531,"thread_start_us":109,"threads_started":1}
I20260811 07:43:26.971877 11117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } }
I20260811 07:43:26.972169 10950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97: Neither blocks nor log segments found. Creating new log.
I20260811 07:43:26.972179 11117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d792be4c96949a7be56319eaeb99fb3, State: Initialized, Role: FOLLOWER
I20260811 07:43:26.972543 11117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } }
I20260811 07:43:26.973304 11125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:26.973439 11125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:26.973958 11117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:26.975128 11034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68: Bootstrap starting.
I20260811 07:43:26.978152 11034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68: Neither blocks nor log segments found. Creating new log.
W20260811 07:43:26.978943 11138 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:43:26.979027 11138 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:43:26.979022 11034 log.cc:824] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:26.979840 10950 log.cc:824] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97: Log is configured to *not* fsync() on all Append() calls
I20260811 07:43:26.977813 11041 master_runner.cc:186] Error getting master registration for 127.18.72.254:38815: OK, instance_id { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" instance_seqno: 1786434206489289 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:43:26.981068 11041 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:43:26.981653 11034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68: No bootstrap required, opened a new log
I20260811 07:43:26.985177 10950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97: No bootstrap required, opened a new log
I20260811 07:43:26.985816 11034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } }
I20260811 07:43:26.986065 11034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89774c15dd6343c3bb0c880aa2370c68, State: Initialized, Role: FOLLOWER
I20260811 07:43:26.986421 11034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } }
I20260811 07:43:26.987022 11142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:26.987320 11142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:26.987787 11034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:26.988296 10950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } }
I20260811 07:43:26.988514 10950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e54ad7b8b54ba3bd8d05cd6f07ba97, State: Initialized, Role: FOLLOWER
I20260811 07:43:26.988838 10950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } }
I20260811 07:43:26.989732 11145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:26.989996 11145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:26.990151 10950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:43:26.990214 10923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 07:43:26.990300 10923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:26.991607 10923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } }
I20260811 07:43:26.992596 10923 leader_election.cc:288] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [CANDIDATE]: Term 1 election: Requested vote from peers 89774c15dd6343c3bb0c880aa2370c68 (127.18.72.253:43999), 1d792be4c96949a7be56319eaeb99fb3 (127.18.72.252:42479)
I20260811 07:43:26.993055 11007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "89774c15dd6343c3bb0c880aa2370c68"
I20260811 07:43:26.993290 11007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:26.994338 11007 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98e54ad7b8b54ba3bd8d05cd6f07ba97 in term 1.
I20260811 07:43:26.991605 10957 master_runner.cc:186] Error getting master registration for 127.18.72.254:38815: OK, instance_id { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" instance_seqno: 1786434206489289 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:43:26.994771 10895 leader_election.cc:302] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89774c15dd6343c3bb0c880aa2370c68, 98e54ad7b8b54ba3bd8d05cd6f07ba97; no voters:
I20260811 07:43:26.994889 10957 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:43:26.995373 11088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1d792be4c96949a7be56319eaeb99fb3"
I20260811 07:43:26.995595 11088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:43:26.996975 11088 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98e54ad7b8b54ba3bd8d05cd6f07ba97 in term 1.
I20260811 07:43:26.998857 11145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:43:26.999076 11145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [term 1 LEADER]: Becoming Leader. State: Replica: 98e54ad7b8b54ba3bd8d05cd6f07ba97, State: Running, Role: LEADER
I20260811 07:43:26.999387 11145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } }
W20260811 07:43:27.000859 11156 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:43:27.000933 11156 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:43:27.001513 10878 master_runner.cc:263] No leader master found from master 98e54ad7b8b54ba3bd8d05cd6f07ba97
I20260811 07:43:27.001578 10878 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:43:27.001878 11145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98e54ad7b8b54ba3bd8d05cd6f07ba97. Latest consensus state: current_term: 1 leader_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:27.001983 11145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:27.002390 11179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:43:27.005180 11179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:43:27.010859 11088 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [term 1 FOLLOWER]: Refusing update from remote peer 98e54ad7b8b54ba3bd8d05cd6f07ba97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:43:27.011521 11176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:43:27.015825 11125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98e54ad7b8b54ba3bd8d05cd6f07ba97. Latest consensus state: current_term: 1 leader_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:27.015959 11125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:27.016925 11007 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [term 1 FOLLOWER]: Refusing update from remote peer 98e54ad7b8b54ba3bd8d05cd6f07ba97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:43:27.017488 11176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:43:27.020718 11142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98e54ad7b8b54ba3bd8d05cd6f07ba97. Latest consensus state: current_term: 1 leader_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:27.020845 11142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:27.024757 11176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:27.024893 11176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:27.026042 11125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:27.026156 11125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:27.026556 11183 mvcc.cc:205] Tried to move back new op lower bound from 7317234511909154816 to 7317234511873343488. Current Snapshot: MvccSnapshot[applied={T|T < 7317234511909154816}]
I20260811 07:43:27.026933 11184 mvcc.cc:205] Tried to move back new op lower bound from 7317234511909154816 to 7317234511873343488. Current Snapshot: MvccSnapshot[applied={T|T < 7317234511909154816}]
I20260811 07:43:27.028301 11145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:27.028406 11145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97 [sys.catalog]: This master's current role is: LEADER
I20260811 07:43:27.029357 11142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98e54ad7b8b54ba3bd8d05cd6f07ba97" member_type: VOTER last_known_addr { host: "127.18.72.254" port: 38815 } } peers { permanent_uuid: "89774c15dd6343c3bb0c880aa2370c68" member_type: VOTER last_known_addr { host: "127.18.72.253" port: 43999 } } peers { permanent_uuid: "1d792be4c96949a7be56319eaeb99fb3" member_type: VOTER last_known_addr { host: "127.18.72.252" port: 42479 } } }
I20260811 07:43:27.029480 11142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:43:27.029597 11186 mvcc.cc:205] Tried to move back new op lower bound from 7317234511909154816 to 7317234511873343488. Current Snapshot: MvccSnapshot[applied={T|T < 7317234511909154816}]
I20260811 07:43:27.031168 11179 catalog_manager.cc:1347] Generated new cluster ID: 36e2b02fbcd0470883f5af15f9898f0f
I20260811 07:43:27.031235 11179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:43:27.042704 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:27.001699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47112 (local address 127.18.72.253:43999)
0811 07:43:27.001817 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.001822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.008188 (+ 6366us) server_negotiation.cc:407] Connection header received
0811 07:43:27.008394 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.008397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.008445 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.008496 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.012445 (+ 3949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:27.013228 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:27.018343 (+ 5115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:27.018714 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:27.018792 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:27.032025 (+ 13233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:27.032030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:27.032033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:27.032061 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:27.033885 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:27.039137 (+ 5252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:27.039142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:27.039144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:27.039248 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:27.039973 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:27.039977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:27.039979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:27.040209 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:27.040331 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:27.042401 (+ 2070us) server_negotiation.cc:299] Negotiation successful
0811 07:43:27.042531 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 07:43:27.044562 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:27.002442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.18.72.252:42479)
0811 07:43:27.002545 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.002549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.009717 (+ 7168us) server_negotiation.cc:407] Connection header received
0811 07:43:27.010121 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.010124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.010177 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.010229 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.014471 (+ 4242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:27.015226 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:27.019582 (+ 4356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:27.019966 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:27.020048 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:27.034456 (+ 14408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:27.034460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:27.034462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:27.034491 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:27.036317 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:27.039564 (+ 3247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:27.039568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:27.039570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:27.039657 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:27.041019 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:27.041023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:27.041025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:27.041256 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:27.041381 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:27.044277 (+ 2896us) server_negotiation.cc:299] Negotiation successful
0811 07:43:27.044406 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 07:43:27.045384 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:27.003050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.18.72.254:38815)
0811 07:43:27.003163 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.003167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.010040 (+ 6873us) server_negotiation.cc:407] Connection header received
0811 07:43:27.011381 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.011384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.011431 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.011480 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.015989 (+ 4509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:27.016742 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:27.021827 (+ 5085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:27.022164 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:27.022250 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:27.036852 (+ 14602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:43:27.036856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:43:27.036859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:43:27.036888 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:43:27.038640 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:27.040409 (+ 1769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:27.040412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:27.040414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:27.040496 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:43:27.041670 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:43:27.041673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:43:27.041675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:43:27.041884 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:43:27.041986 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:27.045089 (+ 3103us) server_negotiation.cc:299] Negotiation successful
0811 07:43:27.045234 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:43:27.063140 11179 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:43:27.064424 11179 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:43:27.087832 11179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98e54ad7b8b54ba3bd8d05cd6f07ba97: Generated new TSK 0
I20260811 07:43:27.088804 11179 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:43:27.114457 10952 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.111289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.18.72.254:38815)
0811 07:43:27.111416 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.111421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.113372 (+ 1951us) server_negotiation.cc:407] Connection header received
0811 07:43:27.113571 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.113575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.113624 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.113675 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.114324 (+ 649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60936: BlockingRecv error: recv got EOF from 127.0.0.1:60936 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 07:43:27.116470 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.112773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.18.72.253:43999)
0811 07:43:27.112887 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.112891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.115125 (+ 2234us) server_negotiation.cc:407] Connection header received
0811 07:43:27.115618 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.115621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.115668 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.115717 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.116353 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47114: BlockingRecv error: recv got EOF from 127.0.0.1:47114 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 07:43:27.117076 11123 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.112087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59012 (local address 127.18.72.252:42479)
0811 07:43:27.112193 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.112197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.114930 (+ 2733us) server_negotiation.cc:407] Connection header received
0811 07:43:27.115393 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.115396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.115443 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.115493 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.116966 (+ 1473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59012: BlockingRecv error: recv got EOF from 127.0.0.1:59012 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 07:43:27.121618 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.119930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47128 (local address 127.18.72.253:43999)
0811 07:43:27.120030 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.120033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.120700 (+ 667us) server_negotiation.cc:407] Connection header received
0811 07:43:27.120912 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.120915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.120986 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.121034 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.121529 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47128: BlockingRecv error: recv got EOF from 127.0.0.1:47128 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 07:43:27.126425 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.125050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47140 (local address 127.18.72.253:43999)
0811 07:43:27.125142 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.125146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.125618 (+ 472us) server_negotiation.cc:407] Connection header received
0811 07:43:27.125786 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.125789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.125836 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.125884 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.126343 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47140: BlockingRecv error: recv got EOF from 127.0.0.1:47140 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 07:43:27.131650 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.130309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.18.72.253:43999)
0811 07:43:27.130394 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.130398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.130834 (+ 436us) server_negotiation.cc:407] Connection header received
0811 07:43:27.130994 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.130997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.131056 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.131101 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.131571 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47148: BlockingRecv error: recv got EOF from 127.0.0.1:47148 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260811 07:43:27.138963 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.137545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.18.72.253:43999)
0811 07:43:27.137660 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.137664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.138108 (+ 444us) server_negotiation.cc:407] Connection header received
0811 07:43:27.138303 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.138306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.138365 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.138413 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.138885 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47150: BlockingRecv error: recv got EOF from 127.0.0.1:47150 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 07:43:27.150947 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.149317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.18.72.253:43999)
0811 07:43:27.149417 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.149423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.150050 (+ 627us) server_negotiation.cc:407] Connection header received
0811 07:43:27.150239 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.150242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.150295 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.150345 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.150852 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47156: BlockingRecv error: recv got EOF from 127.0.0.1:47156 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 07:43:27.171504 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.169675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47166 (local address 127.18.72.253:43999)
0811 07:43:27.169819 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.169823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.170429 (+ 606us) server_negotiation.cc:407] Connection header received
0811 07:43:27.170657 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.170660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.170719 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.170791 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.171388 (+ 597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47166: BlockingRecv error: recv got EOF from 127.0.0.1:47166 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 07:43:27.207628 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.206300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47170 (local address 127.18.72.253:43999)
0811 07:43:27.206430 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.206434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.206694 (+ 260us) server_negotiation.cc:407] Connection header received
0811 07:43:27.206905 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.206908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.206958 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.207049 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.207509 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47170: BlockingRecv error: recv got EOF from 127.0.0.1:47170 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 07:43:27.275213 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.273881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47184 (local address 127.18.72.253:43999)
0811 07:43:27.274044 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.274049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.274189 (+ 140us) server_negotiation.cc:407] Connection header received
0811 07:43:27.274379 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.274382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.274455 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.274563 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.275081 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47184: BlockingRecv error: recv got EOF from 127.0.0.1:47184 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 07:43:27.406595 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.405389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.18.72.253:43999)
0811 07:43:27.405543 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.405548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.405689 (+ 141us) server_negotiation.cc:407] Connection header received
0811 07:43:27.405875 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.405878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.405927 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.406016 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.406492 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47196: BlockingRecv error: recv got EOF from 127.0.0.1:47196 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 07:43:27.666446 11040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:43:27.665123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47208 (local address 127.18.72.253:43999)
0811 07:43:27.665281 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:27.665286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:27.665446 (+ 160us) server_negotiation.cc:407] Connection header received
0811 07:43:27.665629 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:27.665637 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:27.665689 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:27.665780 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:43:27.666300 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47208: BlockingRecv error: recv got EOF from 127.0.0.1:47208 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 07:43:27.980438 11138 catalog_manager.cc:1259] Loaded cluster ID: 36e2b02fbcd0470883f5af15f9898f0f
I20260811 07:43:27.980525 11138 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3: loading cluster ID for follower catalog manager: success
I20260811 07:43:27.982131 11138 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3: acquiring CA information for follower catalog manager: success
I20260811 07:43:27.983227 11138 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d792be4c96949a7be56319eaeb99fb3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:43:28.002118 11156 catalog_manager.cc:1259] Loaded cluster ID: 36e2b02fbcd0470883f5af15f9898f0f
I20260811 07:43:28.002179 11156 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68: loading cluster ID for follower catalog manager: success
I20260811 07:43:28.003695 11156 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68: acquiring CA information for follower catalog manager: success
I20260811 07:43:28.004810 11156 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89774c15dd6343c3bb0c880aa2370c68: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:43:28.184937 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:28.180916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.18.72.253:43999)
0811 07:43:28.181193 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:28.181197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:28.181216 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:43:28.181363 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:28.181366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:28.181505 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:28.181628 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:28.181960 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.182546 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.183391 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.183635 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.183721 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:28.184081 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:28.184552 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:28.184654 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:28.184741 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 07:43:28.184806 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":107,"threads_started":1}
I20260811 07:43:28.192593 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:28.188121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59026 (local address 127.18.72.252:42479)
0811 07:43:28.188373 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:28.188376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:28.188388 (+ 12us) server_negotiation.cc:407] Connection header received
0811 07:43:28.188468 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:28.188470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:28.188672 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:28.188786 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:28.189013 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.189697 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.190697 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.191011 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.191178 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:28.191484 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:28.192025 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:28.192173 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:28.192321 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 07:43:28.192389 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":99,"threads_started":1}
I20260811 07:43:28.200340 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:28.195998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59028 (local address 127.18.72.252:42479)
0811 07:43:28.196110 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:28.196114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:28.196749 (+ 635us) server_negotiation.cc:407] Connection header received
0811 07:43:28.196853 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:28.196856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:28.196910 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:28.196987 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:28.197345 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.198040 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.199121 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.199430 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.199499 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:28.199834 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:28.199939 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:28.200010 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:28.200144 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 07:43:28.200195 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 07:43:28.200436 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:28.196018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47224 (local address 127.18.72.253:43999)
0811 07:43:28.196171 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:28.196176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:28.196320 (+ 144us) server_negotiation.cc:407] Connection header received
0811 07:43:28.196518 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:28.196521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:28.196584 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:28.196658 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:28.197017 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.197805 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.199105 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.199412 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.199537 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:28.199875 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:28.199976 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:28.200051 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:28.200221 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 07:43:28.200293 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:43:28.201186 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:43:28.195625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.18.72.254:38815)
0811 07:43:28.195869 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 07:43:28.195874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:43:28.195911 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:43:28.196100 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:43:28.196103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:43:28.196276 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:43:28.196417 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:43:28.196662 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.197540 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.198858 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:43:28.199209 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:43:28.199333 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:43:28.200001 (+ 668us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:43:28.200385 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:43:28.200487 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:43:28.200967 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 07:43:28.201037 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260811 07:43:28.204717 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 10878
I20260811 07:43:28.215250 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 10957
I20260811 07:43:28.223404 18723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskIvL2Zc/build/debug/bin/kudu with pid 11041
2026-08-11T07:43:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3112 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6244 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287256 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST